The solution: Add another vanishing point (VP) and treat it separately from the first one.
In this example, the houses in the foreground follow one VP, and the houses in the background (after the bend) follow another, because their orientation is slightly different.