Is the current solution for Route Deviation provided by Flex sufficient?

[westontrillium]

Describe the problem

My experience so far with complete producer-consumer Flex implementations has been that route deviation is exponentially more complex than any other use case. The current approach is to explicitly define polygons representing each pickup/drop-off zone between every stop-to-stop segment in a trip and then explicitly define their pickup/drop-off windows based on the timepoints they are sandwiched by. This is a very heavyweight approach due to the amount of data processing required both by the producer and consumer. One specific example is how it creates many geographical overlaps in small spaces which poses challenges for querying actual deviation trips. The problem increases the denser the stops/shape areas are. Implementing on the producer side is equally burdensome, especially when we consider the fact that route deviation is a modification of existing fixed routes and should lend itself well to a lighter-weight extension. Currently, if a producer without data automation tools wants to implement route deviation, they would need to draw polygons for each of those stop-to-stop segments, which would require at least a working knowledge of GIS (even the shortcuts are predicated on being proficient enough in GIS to know what those shortcuts are). That producer would also need to manually enter in each start/end pickup/drop-off window based on the surrounding stops (it's also unclear what should happen for cases for stops without arrival/departure times).

Use cases

Fixed routes that can deviate within a pre-defined boundary off the regular path of travel to pick up or drop off riders upon request.

Proposed solution

At first one might assume that adding a component specific to route deviation would not be backwards compatible with the Flex components already introduced into the spec. In reality, nothing in the current spec would actually contradict additional parameters specific to route deviation, because route deviation is currently handled by applying "route category agnostic" components of Flex to model the use case (i.e., there is nothing specific to route deviation in the spec).

One solution I've thought of would be one that does not utilize locations.geojson or the time windows at all but rather adds "metadata" fields to stop_times.txt. This is actually similar to how route deviation was handled in the original Flex specification but with slightly different rules, especially in light of the already-existing Flex components in the spec.

stop_times.txt (extended)

Field Name	Type	Presence	Description
pickup_deviation_radius	Non-negative float	Conditionally forbidden	Radial distance in meters from the trip's shape between this stop and the following stop. This value indicates how far the vehicle may deviate from this segment to pick up a rider. A deviated pickup within this radius can only occur after this stop's departure_time and before the following stop's arrival_time. The trip must specify a shape_id for this field to be defined. Forbidden if this is the last stop of the trip. Optional otherwise.
drop_off_deviation_radius	Non-negative float	Conditionally forbidden	Radial distance in meters from the trip's shape between this stop and the following stop. This value indicates how far the vehicle may deviate from this segment to drop off a rider. A deviated drop-off within this radius can only occur after this stop's departure_time and before the following stop's arrival_time. The trip must specify a shape_id for this field to be defined. Forbidden if this is the last stop of the trip. Optional otherwise.
deviated_pickup_type	Enum	Conditionally required	0 or empty - No deviated pickup available. Forbidden if pickup_deviation_radius is defined. 1 - Must phone agency to arrange a deviated pickup. Required if pickup_deviation_radius is defined.
deviated_drop_off_type	Enum	Conditionally required	0 or empty - No deviated drop-off available. Forbidden if drop_off_deviation_radius is defined. 1 - Must phone agency to arrange a deviated drop-off. 2 - Must coordinate with driver to arrange a deviated drop-off. Value of 1 or 2 is required if drop_off_deviation_radius is defined. Optional otherwise.

This covers the vast majority of route deviation services in the United States operate based off of a fixed radial value ("3/4 of a mile off the fixed route" for example), and even ones in which that radial value fluctuates stop-to-stop. This would not cover unique deviation situations like where the deviation zone is actually a custom area not tied to the shape, however those could still be covered by the agnostic Flex components currently being used to model route deviation.

This is definitely just a draft idea and needs to be further refined, but I think it could be a good start for rethinking how we do trip planning around route deviation long term.

As a completely unqualified thought experiment, I've worked out the kind of logic possibly needed to return a route deviation trip itinerary for a pickup using the above framework.

For pickup:

1. Check for stop with deviation radius defined closest to requested location beyond walk threshold.
2. Check if requested location is within radius value of shape_id between this stop’s and next stop’s shape_dist_traveled segment.
3. Check if requested pickup time is after stop’s departure_time and before next stop’s arrival_time.
4. If there’s a match, then the trip planner displays that stop’s arrival_time for the deviated pickup (potentially with factor/offset applied value) along with any booking_rule information for that stop_time.

Additional information

I would like to hear from consumers on whether such a solution would make it easier to provide better route deviation trip plans. If not, it would be equally valuable to hear other ideas. :)

[leonardehrenfried]

I had a chat with Weston yesterday and would like to summarise my thoughts about this from consumer (or at least OTP) point of view: I see it as a shorthand notation of something that is already possible but doesn't have an explicit name in the spec.

In our implementation at least doing away with the explicit polygons and windows would make things more complicated as we internally need both to generate the routing results. If I were to implement this, I would add a preprocessing step that reads the information and converts to the "polygons and windows" model before it can be handed off to the routing engine.

Weston told me that he thought that it would make consumers' life easier and I'm very sorry to say that it's not the case for OTP.

That said, if it would make the life of a producer much, much easier I'm willing to entertain the idea but I would see it as a concession rather than something I would embrace enthusiastically.

[gcamp]

+1 to what @leonardehrenfried said.

Unless we drop entirely the polygon functionality, a change like this will not help implementation.

Even if I'm sympathetic on the difficulty of creating accurate polygons, could a simple open source script could exactly that? Or the difficulty of using/setup a script would be as difficult as creating the polygon in the first place?

[westontrillium]

Thanks for your comments @leonardehrenfried and @gcamp! As I mentioned to Leonard in our conversation, for this case I am definitely not interested in a solution that would make it easier for one party but harder for the other (and in fact, switching the way we do route deviation would mean more work for us too, at least in the short term. But if it means we'd get vastly better end results, it's worth it in my opinion).

The point of this thought experiment is really to answer in definite terms whether the current approach to route deviation is the best solution to yield reliable trip plan routing using a not-too-unreasonable data scheme, because I don't think that question has been properly asked yet. If the answer is "yes this is the best way," great! Having that established unequivocally would be equally valuable. What I have seen, though, is that deviating routes are much more difficult to get to behave exactly as expected in the trip planning environment. Can that be helped at the spec design level? Or maybe the difficulty of implementation is just an intrinsic reality of route deviation itself? If so, is it worth exploring other ways we might alleviate the difficulty? Establish clearer instructions around data production? Implementation troubleshooting guidelines? Just thinking out loud here. 🙂

[tzujenchanmbd]

I would like to add a perspective regarding the stop_times.txt, perhaps consumers can confirm further.

The stop_times.txt is usually the largest file in a feed. Adding more columns to stop_times may lead to performance issues when parsing the data (especially for those huge feeds). Due to the 6 new columns added to stop_times.txt by Flex, we have already observed some issues while creating new validator rules.

When adding columns to stop_times, we may need to consider performance issues more carefully. (e.g., whether there are alternatives)