Open Data

So, you want people to engage with your organisation and be able to understand what information it is you hold, the role that you’re playing, and you do that through providing your data to other people in a way that makes it simple for them to use and repurpose. Through that process you enable innovation, I suppose, in other parts of your society. And what you’re really chasing, I suppose, is the, the long term value or the societal change that will come about over time as people start to engage with you, use your data, use your information, and create those sort of social and political and economic sort of changes and benefits over time.

It’s not a short game, it’s definitely the long game that we’re playing. The value only comes over time, there’s no good sort of putting out a small bit of data of dubious quality and hoping that you’re going to get significant benefit from that. What you really want is you want to work away at the coalface and understand what good looks like. And so, in Hamilton we are committed to modelling our data. So, all the data that lives in our business systems, we want to then pull that apart and put that back into conventionally named business data models and then ship that as open data so that it becomes as useful as possible at the other end.

What we’re also doing is as we put that data out, we’re consuming it ourselves to produce what we are sort of terming ‘open analytics’ to go alongside the open data. So, that’s where we get to tell our own story using the data and sort of say ‘This is how we would present it but we also want to give you the opportunity of drawing the raw data yourself and you might do something that we haven’t thought of’. So, there’s the innovation part. So, you have to consider all the people involved in that chain of creating the data in the first place, and you have to think about quality. So, we’ve got to focus on trying to build

quality into our platform as we go. [Chris Vas]: Now this data, as I understand it, is not just sitting within one particular Council, this is data that’s sitting across multiple Councils? [Chris Martin]: Right. So, there’s 13 Councils in the Waikato and what we’ve decided to do is let’s not reinvent the wheel numerous times, let’s try and do it together. So, we created one channel for open data or one portal that the 13 Councils will publish their information to. But the internal process of getting data from a particular Council to the portal, that’s an individual journey for each Council. And it needs to be that way because each Council has different systems, different software, and they’re at different stages of maturity.

I think the, the benefit in doing it together is that the more mature Councils or the Councils that have got more time to put into this will do the learning and share that amongst the region. That’s the approach that we’ve taken and that’s why we decided to collaborate. [Chris Vas]: Okay. How did you get 13 Councils to agree on this? [Chris Martin]: That wasn’t easy. But I think there are probably a couple of key areas with that.

It’s notoriously hard to get organisations to collaborate and one of the things that we did early on is we, we had three Councils that said, ‘We’ll devote time to this and we’ll do whatever it takes to shape the opportunity up and to communicate with the others’. So, we took the effort off those Councils that might not have had time initially. The second thing we did was we designed a process or a framework where we said it’s actually not compulsory that you guys contribute to this.

So, we had a whole lot of stages and each stage had an exit ramp and it basically sort of said, ‘Hey, if we can achieve this phase of the, of the project we’ll move on to the next. If we can’t achieve it, we’ve got two decisions. We either go back and re-, redefine that phase and we try and do something that we can all agree to, or we exit the project and we go and put our energy elsewhere.’ And I think that created quite a lot of freedom within the project, people didn’t feel like they were stuck in an endless loop of having to do something. So, and yeah, fortunately it kind of panned out. So, yeah.

So, by moving together as a region, those Councils that might never have sort of got there or considered the problem for some years down the track, they got the opportunity of being able to actually just learn a little bit as we go and, and get ahead of, ahead of the game, so to speak. [Chris Vas]: Has there been any early wins in any particular area of how the community or society has used some of these datasets to improve decisions or come out with new ideas or make changes? [Chris Martin]: So, I think it’s a bit early in the piece for some of those early wins in terms of real innovation.

What we are experiencing is those organisations that like to procure data from us, they’re finding it easier to do that. And that was one of the key drivers from the outset. If you’re looking for data across the region you don’t want to have to go to 13 Councils and try and find the right people to, to talk to. You want to go to one place and kind of have it all there. So. [Chris Vas]: What’s been the relationship with external vendors, you know, folks and institutions that sit outside of the Council remit, particularly private sector organisations that, you know, are technology-based organisations and the others?

Because obviously that, building that innovation ecosystem has to be quite a critical part of this initiative. [Chris Martin]: Obviously open data relies somewhat on technology; we’ve got to have a portal. And so we selected a vendor to work with based on the fact that some Councils were already working with a particular vendor, we had a bunch of selection criteria, but at the end of the day we wanted this thing to be as low-cost as possible. And the reason for that is we didn’t want it to be a barrier to other Councils coming on board.

Since we have launched this though we’ve had a, a, quite a big show of interest from other technology companies who are sort of going, ‘Hey, well, as you, as your open data grows, this will be a great place for us to potentially come and use that information to do other things with’. So, yeah, we’ve had a lot of interest but there’s no real sort of use on the innovation up front – yet. It’ll come. [Chris Vas]: Okay, okay. And so, if you had a crystal ball and, you know, you had to gaze over the next three to five years, what sort of outcomes would you expect to see from this initiative?

The library’s a great space, it’s a public place and there’s a, a big opportunity for us to be able to run a quarterly outreach sessions or those sorts of things, from a space like this. We also do a bit of advertising here and there as new datasets are released. We, we like to advertise those and we always give out our contact details so we’re encouraging people to connect with us. The other thing, I suppose, is that as we set this project up we stood up a customer reference group. So, we went out to the community and we sort of said, ‘If you want to be a part of this please register’.

We ended up with about 180 participating organisations who we have a list of contact details for. We use them to sort of garner responses from. So, we’ll be polling them, ‘Hey, we’ve put this data out, how useful are you finding it?’ And that’s led to a few changes already in the way that we actually post some of our data up. [Chris Vas]: I guess every time we talk about data there’s this issue around privacy. [Chris Martin]: Correct. [Chris Vas]: Has this issue come up from the community at all?

I mean you’ve had significant response of, you know, having 180 organisations respond to your open call… [Chris Martin]: Yep. [Chris Vas]: …was privacy ever a consideration around making data available publicly? [Chris Martin]: Yeah. So, privacy is a key consideration, obviously. And so, this is where, I suppose, you have to put your effort into your framework or your process. So, as I said before, each Council has their own process for getting their data onto the platform. So, ours, we call ‘the open data pipeline’ and it’s a series of gates that we have to go through before we can publish our data.

And one of those gates is what we call a risk assessment, and that covers off things like personally identifiable information, political risk, commercial risk, and those risk of deaggregation, so, people being able to take a dataset and use with other datasets to identify individuals. And we put a little bit of rigour around that and then we engaged the business owner of that data and that process and we make sure that the data we’re putting out gives us the lowest possible risk around privacy. [Chris Vas]: In terms of your open data initiative here in Hamilton, New Zealand, are you looking at other best practices overseas or across the country to form some of your thinking and your approach here?