Open Knowledge group presenting a talk by VicRoads, hosted at ThoughtWorks, 4/16/2015:
this transcript is made straight from the notes I took at the event – it is definitely missing some comments, and quite likely there are mistakes in what I did capture. If you see any, comment here or email me and I will fix them.
Three guys (Adrian Porteous, Evan Quick, Phil Reid) from VicRoads talking about their open data – current, planned, and what people would want in the future.
Steve Bennett, from Open Knowledge, who has worked with VicRoads and has many open data mapping projects – http://stevebennett.me/, http://cycletour.org/, http://www.opentrees.org/, openbins.org, opencouncildata.org and melbdataguru.tumblr.com
In 2013-ish the Victorian government created a mandate to make government data open where possible. VicRoads specifically had already been doing this, with crash data etc. Public Transit Victoria is under heavy pressure to make their data available in the General Transit Feed Specification (GTFS) format for eg: Google Maps to use, and VicRoads to do real-time traffic data. Unfortunately they don’t currently have the infrastructure to serve everything, inventories of what they have available, or even a good understanding of what data would be useful to the public.
Their focus right now is on:
- real-time traffic data
- Integrating the released data with spatial data
- Releasing an inventory of available (and externally relevant) data
Steve was seconded from his position at the University of Melbourne for six weeks in December to start looking at what they had and help them figure out what to do. While there, he found one pretty cool dataset: every two years, VicRoads runs cars with cameras along every road they own (freeways and main roads in Victoria) to get a Google Street View-like set of images, which they use in the office to decide on maintenance priorities, etc. They were looking at this dataset wondering if they could somehow automate road sign recognition in it to help with inventory management (road signs are all owned by VicRoads, including every speed zone sign in the state). Steve said that they could actually publish this complete dataset, about 2.5 terabytes worth, on VicNode. (VicNode is a service for hosting research data by scientists across Victoria: Steve also works with them). Once hosted there, they could submit it to Mapillary, a crowdsourced Google Street View analog. This started extensive discussions about who owned the dataset, what restrictions there were on sharing it, etc – but now it’s on track to be on Mapillary this week (update: approval came through and the uploading is in process!). This is a great example of how they hope getting open-data people involved will help them recognise data that can be published and find ways to do it easily. Part of the reason they are here giving this talk is to solicit ideas from people here about what they want from VicRoads – what kind of data do they want, what quality do they need, what format would be useful, etc.
Question time!
Q. How does real-time traffic data collection work?
A. Every traffic signal has sensors, mainly to make adaptive light timing work. There are 6-12 detectors per approach at an intersection.
Q. So collection on rural roads [that have no traffic lights] is not as good?
A. Correct. They also do a lot of manual collection, where you lay a cord across the road and count them that way, and all freeways have detectors every 500m. This was all set up for traffic management purposes, but once the data was being collected they started using it for other things. They’ve also set up bluetooth sensors – the BT sensor sees your phone broadcasting a bluetooth ID at the start of a collection path, and then another sensor x miles later sees you again and they aggregate that data into trip time information. Also, the entire taxi fleet has GPS data that VicRoads can track.
Q. What about traffic measurement on bike routes?
A. Yes, we have traffic sensors on 30-40 of the major bike routes. Also, there are truck weight sensors in the roads that can count the number of axles on a truck as it goes over.
Q. Do these sensors catch every car going over?
A. The sensors themselves do, but we only look at aggregated data.
Q. What’s the timing of the release of CrashStats data? [https://www.data.vic.gov.au/data/dataset/crash-stats-data-extract: asked by one of the guys who runs http://www.crowdspot.com.au/ – he asks because they tried to work with this data and it seems to be delayed release]
A. There is fairly complex procession around the CrashStats data. VicRoads get it from the Victoria Police for crashes that caused injuries or fatalities, basically just as an incident report. (The Victoria Police incident report data is also published by TAC. VicRoads do also have RACV towtruck calls data for crash responses, which is less complete and not used in CrashStats?). Once the incident report is received, they investigate to identify the cause of the crash, the eventual outcome (delayed fatalities or recoveries) and once that’s finalised, they publish it to the internal dataset. This includes details like the angles and speeds of cars involved, crash diagrams, etc – any identifying details are removed before public release. Data is published to a semi-private system (available to councils) six-monthly. The public data host was built as a student project several years ago and looks like it. They are currently planning a new version and trying to decide what it needs.
Q. Are hospital records included?
A. No.
They don’t really capture details of accidents with no injuries reported – perhaps they should, perhaps it would show that those crashes are leading indicators of where fatal crashes will be. They are resource constrained on all this (nobody has a day job of releasing VicRoads data to the public) but are working hard to make the data more available. So they put a dump on 6 years of raw data on data.vic.gov.au last year for GovHack, but haven’t kept updating it. They are embarrassed.
Q. So, weighing vehicles. I live in a residential area and we get a lot of trucks coming down back streets. Do you know where they go?
A. Through the Intelligent Access program, they actually get the GPS records from most trucks. Theoretically they then get checked and told not to go off allowed routes – but in practice they mostly only get followed up on for going over bridges that aren’t rated to support their weight. The Open Access data isn’t open (it is a federal program and probably the data is owned by the federal government?) but it would be awesome if it were.
Q. So those truck weight sites don’t tell you?
A. Actually they know that trucks deliberately avoid the weight sites, which are all public knowledge, because the amount of traffic they see is lower than the amount of traffic we know exists – but they don’t know where they go instead except by looking at the Fed data. If this were opened up, it’d probably be possible to get some neat matches between the two datasets using linkage mapping.
All speed signs in Victoria must be approved by VicRoads (not the case in all states), and the dataset of all speed zones is available on data.vic.gov. They are trying to get that dataset into a realtime update so that e.g. the variable speed on the Westgate Bridge is reflected online immediately.
Q. Is there any data on whether former blackspots (high accident intersections) are safer after speed zone changes?
A. There probably is, but these guys don’t know it. The Monash Accident Research Center is involved? Anecdotally, they think the data showed it didn’t have a huge impact and that’s why the program wasn’t continued.
VicRoads is considered a leading example of government departments providing open data, but these guys feel skeptical because they don’t feel like they are doing such a great job – for instance, they have 650 internal datasets, and only about 40 have been published as open data. The unreleased datasets include stuff like
- bridge height measurements, could be used for intelligent routing of trucks. Currently there are FIVE bridge strikes EVERY DAY in Victoria, which requires traffic management, possibly bridge closures and repairs, etc etc.
- maximum bridge crossing weight, ditto
so if you have a specific idea of data that would be useful to you, they might just have it already! There is a ‘suggest a dataset’ option on data.vic.gov, or you can email these guys directly to kickstart the process.
Back when they started doing open data, it wasn’t really run well: so they had quotas on how many unique datasets needed to be released, but no quality control, so people released silly stuff like an entire dataset consisting of 4 values. These days, they still have issues like liability concerns on releasing data, e.g. if the bridge height data is released and contains an incorrect value and someone’s intelligent routing relies on this and directs a truck under a bridge that it hits – who is liable, for what? And this data does change, each time a road is resurfaced it rises by like an inch so the bridge clearance decreases, if the clearance hasn’t been measured in a decade it could be several inches off.
Q. Do you have any datasets that could be crowdsourced for fixes to make them more complete/accurate?
A. They don’t have any plans (or means with which) to do so atm, although it’s a pretty interesting idea to explore in the future. IMHO it’s something that could be driven by the open data community.
Q. Is it conceivable to provide the data from traffic sensors as a realtime raw feed, or other sensors?
A. It depends on the data. Road closure information is up to the minute already, but parking spot sensors are delayed two weeks. [Questioner wants timelapse traffic models of particular streets at different times of day to evaluate development decisions]. So some of it might not be realtime, but we have built an app that shows average traffic for any road at a chosen time of day, which is already publicly available. (??Didn’t catch where this is, couldn’t find it online).
Q. On those five bridge strikes a day that were mentioned earlier: is that data available anywhere?
A. No, actually. Someone must collect it, for instance there is a specific bridge strike response team – but they just write up freeform reports and there’s no system that parses those for the bridge name/location etc.
Currently on data.vic.gov: mostly the high-interest data sets like heavy vehicle information (allowed routes vary by weight of truck, size, day, time of day, etc – this is all used by trucking companies), speed sign locations, and crash stats (available both as a java app and a raw dump) which covers all accidents with injuries since 2005 (internally they have this data back to 1986). They recognize that they aren’t doing well at keeping released datasets up to date, and this is one reason they are interested in ArcGIS Online (vicroadsopendata.
There is a Road Use Priority dataset showing which population a road is primarily intended to serve – eg, truck routes, bike routes, pedestrian areas. This is an example of how VicRoads has shifted it’s purpose from road management to transport systems management.
Q. Is that traffic data for bikes available?
A. Yes, on data.vic.gov, and they are about to release a bike traffic modeller like the car traffic modeller mentioned above.
Q. Do traffic light sensors count bikes?
A. Hm, they’re not sure. Anecdotally, bikes in car lanes trigger traffic lights – they definitely have sensors in bike lanes and also in some of the main trails around Melbourne.
Q. Could residents on a backroad ratrun volunteer to pay for e.g a bluetooth sensor that will track how many cars are going through, to get it looked at?
A. Actually, councils already have temporary counters available – when a complaint is made about inappropriate traffic, the council installs it (costs a couple hundred dollars to run for a couple weeks), then looks at the collected data and decides whether there is a real problem that needs traffic management. So all these councils have small collections of data about back streets, and VicRoads has started wondering if they can do something with it.
They don’t currently have a way to aggregate data from multiple councils (but opencouncildata.org is trying to work on this), or from any crowdsourced data collection. They probably won’t get around to this, and an ideal solution is that VicRoads releases what they have, the councils release what they have, and the internet aggregates it all.
edited with some corrections from Steve 4/23