Jon Blower
Chief Technology Officer at the Institute for Environmental Analytics. We help all kinds of people to understand and make the most of environmental data!
Location Reading, United Kingdom
Activity
-
Jon Blower replied to John Norris
Yes, the common use of the term "API" has shifted a bit recently. Quite often when people say "API" these days, they mean a means to access data over the web, in a machine-readable way. These kinds of APIs are very important for accessing and integrating different kinds of data. (Commonly data may be supplied in a standard format such as JSON.)
In order to...
-
Good spot, I will raise this!
-
Jon Blower replied to Daniel Völkl
Yes, both good points. Transparency is a big motivation for Open Data. And people do worry a lot about misinterpretation, and this has often been a justification for not publishing data. My own opinion is that, for most public datasets, the benefits of transparency and trust outweigh the worries over misinterpretation, which I think are often exaggerated. (If...
-
Jon Blower replied to Rastislav Rybanič
Welcome to the course! The use of Big Data to guide policymaking is extremely important. In the UK we have a 25-year Environment Plan, which will shape this area (https://www.gov.uk/government/publications/25-year-environment-plan). We'd love to use our data and expertise to contribute towards the plan, and it's a big challenge to work out how best to do this...
-
Jon Blower replied to Niall Geraghty
Absolutely right!
-
I agree too! The importance of storytelling is much overlooked. People generally respond well to narrative explanations
-
Jon Blower replied to Niall Geraghty
Great discussion. To add to this, there has recently been a move towards providing satellite data in "analysis-ready" forms. The idea is that some of the complexity of the data is removed, by doing necessary corrections and regularisation of the data up-front, thereby making it easier for a wider community to access. If you do a web search for "Analysis Ready...
-
Jon Blower replied to Eduardo Martín
@NiallGeraghty definitely, data ought to be provided in machine-readable forms (not PDFs - agh!). But when we are considering data volumes in the terabyte and petabyte scale, we can't give it all to the user. There's a big challenge in working out how much is "enough" for different user types.
-
Jon Blower replied to Eduardo Martín
Great point. It's certainly true that data are sometimes technically "open" but in practice are still very difficult (and hence expensive in terms of time) to use. Lots of people are working on this, but I personally think that "usability" is an aspect of open data that hasn't been looked at enough.
-
Jon Blower replied to andres V
Nice example - I bet many areas of the sports industry are ahead of most other sectors in terms of data analytics
-
Jon Blower replied to Jim Hill
Hi Jim, this is a really great point. At the IEA we are working on a renewable energy planning platform aimed at developing island states. Slow internet speeds and lack of access is certainly a problem. (It can also still be a problem in the "first world", in rural or inaccessible areas.) This is where cloud computing can play a role, to do all the storing and...
-
Jon Blower made a comment
Wonderful to see so many learners from so many different countries! I hope you all enjoy the course.
-
Nice set of applications here!
-
Yes. There are lots of examples of using satellite data in non-commercial (and commercial) settings for environmental improvement. Satellite imagery remains the best way of monitoring large-scale deforestation, for example. It can be used to provide evidence to ensure that landowners and farmers adhere to policies (e.g. on allowing land to lie fallow and...
-
You make a very valid point about the possible drawback. It's interesting that people react to computer-based systems like this in different ways. The feedback we've had on this system has been very positive overall - I think most people like to see this kind of thing more than reading the same information in lengthy reports!
-
Jon Blower made a comment
Great suggestions everyone!
-
Jacqueline - this is very nicely put. I think you have grasped the main point, which is that visualisation is not just about "pretty pictures" (dissemination) but also about aiding discovery.
-
I think it depends on how you use it. It may be easier to think of visualisation as an *act* rather than a *thing*. For example "the visualisation task I am doing now is observational in nature", rather than "this thing is an observational visualisation". Does that make sense?
Maybe my answer to "Neil AT" above might help answer this too?
-
The way I see it personally is this - if you're just making some kind of a plot, without trying to answer a particular question, this is "observational visualisation". To me, it just means "taking a look at the data and getting a feel for it". What is the resolution? Where are the gaps? Is there clear structure in the data or is it "noisy"? It's often the...
-
To give an example: imagine that I look at my thermometer in my garden and it says the temperature is 50 degrees Celsius. Do I believe it? Probably not, because my (mental) model of the world tells me that the temperature is never this high in March in the UK. My model is correcting my observation. In reality, we combine many sources of information with models...
-
And to complicate things further, don't forget that data (observations) also have errors. Introducing a (good) model can actually reduce the errors. And you need some kind of a model (simple or complex) to give you an estimate of what is going on *between* the observations, i.e. where you have no data. A key point is to understand how much *information* there...
-
Really interesting points. There are so many types of model it's hard to know where to start. A model is just a view of the way something works - a relationship, or set of relationships, between things we observe. It might be something really simple (like a linear relationship between two things) or something extremely complicated (like a weather forecast,...
-
Great point. I think the increased provision of "Analysis Ready Data" is a big plus point for users. Of course, some people will still want to go back to the raw data if they have very specialist applications, but ARD opens data up to a much wider audience.
-
Jon Blower replied to Richard Dennehy
Absolutely - and I think this is really the core of "data science".
-
Jon Blower replied to Murdoch Baxter
Any kind of data storage is a trade-off between the cost of storage and the speed of retrieval. Tape is very cheap for storage but slow to retrieve - but it may be the only viable option for very large archives that we don't need to access very often. (Tapes are much better than they used to be though!)
-
Jon Blower replied to Fiona Potts
Great points - data storage and processing are not free, either in terms of cost or environmental impact.
-
Jon Blower replied to Anahí Castañeda
"Veracity" means reliability, in the sense of accuracy. "Velocity" means how fast the data are collected or transferred.
-
Yes, there is a big issue of communication and social science (how do people react to evidence) as well as generating the evidence from data.
-
Well I can't speak about economic models, but we *can* predict the weather, albeit imperfectly. We can show that average weather forecast accuracy (known as "skill") is increasing with developments in data acquisition and modelling. You are absolutely right that it is very hard to get a grip on the uncertainties of complex models, and this is a real challenge....
-
Yes - increasingly commercial vessels have sensors for lots of things, including weather and current. And scientists have attached other instruments to ferries and other vessels to measure many things (these are called "ships of opportunity").
-
Great point. Some things (like air pollution and noise) are very challenging to measure because they vary so much on small spatial scales. A measurement in one place could be very different from a measurement only a few metres away.
-
It's a great point that just because we have *lots of* data, doesn't mean we always have the *right* data for a problem.
-
Jon Blower replied to Uche Ugwanyi
It's hard to compare the capability of a big computer with that of the human brain because they don't work in quite the same way. Your brain can do plenty of things that even the most powerful computers can't (yet)! One example is recognising familiar faces, which humans do better than computers. (Unfamiliar faces are a different story...)
-
Jon Blower replied to Richard Dennehy
The weather forecast *is* getting more accurate, but the weather is a chaotic system that can never be predicted with 100% accuracy. A crucial part of getting the weather forecast right is understanding what is happening *now*. This is where data comes in. We use data from satellites, weather stations and other sources to get the best picture of the current...
-
Same here - I hadn't heard of it but will check it out!
-
Jon Blower replied to Matt Scott
Hi Cristina - we do not have observations of everything we would like, unfortunately. If we had a higher density of sensors we could certainly do a lot more. There is a lot of interest at the moment in deploying sensor networks in different locations, particularly cities ("Internet of Things" is the key buzzphrase!). There is a cost to this of course, both in...
-
Jon Blower replied to Matt Scott
Hi Cristina - the problem is one of scale. Fog can be caused by highly local conditions, which are not always detectable by the information we have on weather. If we had extremely fine-grained weather information it would be easier to use this approach.
-
I hadn't heard of brontobytes or geopbytes! We'll have to update the infographic...
-
Jon Blower replied to Alastair Macrae
Yes, that's right, which is why it's often not worth the effort to delete it. (Because you would need to develop and implement procedures to decide what to keep.) But it's worth noting that sometimes we *should* delete old data, e.g. if it is personal in nature and no longer relevant (because of Data Protection).
-
That's a very interesting case. Last week I came across a new startup company in Portugal (BitCLIQ), who are using blockchain technology to digitally assure the traceability of seafood products. I'm sure the same ideas could be applied to the supply of any goods, including aircraft parts.
-
Jon Blower replied to Vanessa Yuen-Roberts
It's worth noting that you need a licence even if the data are made available for free to everyone (e.g. a Creative Commons licence). A licence does not necessarily mean that money changes hands. However, sometimes the choice of licence is governed (or informed) by the original conditions of the funding that led to the research.
-
Jon Blower replied to Alastair Macrae
Vanessa - interesting idea. One issue in research is that the value of the data may not be known for a very long time after collection. And we already have to make some tough decisions about how much data to keep. But I do like the idea that there could be a model in which costs are deferred until the value is better known. It's a bit like getting a loan...
-
Jon Blower replied to Alastair Macrae
Nuno - I saw an interesting presentation from a data centre a while ago, where they gave another reason why they keep everything. If the archive is growing exponentially, then there is little point in deleting old data, as it represents such a small fraction of the archive. It's easier just to keep everything!
-
Great example - I really like the way it starts with a "story", but then lets you explore the data in your own way
-
Firstly, congratulations on getting to grips with Python/R/etc! It's not an easy thing to move from something like Excel to a programming language, but it gives a massive increase in power and flexibility. Regarding data size, having data that are "too big to download" is a useful working definition of a Big Data problem! If the data provider doesn't provide...
-
Absolutely - "domain knowledge" is very important. Sometimes this means you need assistance from someone else, who knows the domain best.
