Vicky Lucas

Vicky Lucas

Training manager for the IEA - interested in all things about learning and the environment.

Location UK

Activity

  • Hello and welcome to the course - there will be four of us actively contributing over the next three weeks - myself, Sally, Jon and Liam.

  • Welcome to the course! We have an article on a data visualisation which is part of a renewable energy project for Seychelles in Week 3.

  • We'll be looking at smart meter data in Week 2

  • Large datasets are a challenge for storage and analysis. There are several projects (such as this example from Europe https://www.copernicus.eu/en/access-data/dias) which offer cloud computing to help remove this burden from the user - one step in the right direction.

  • Excellent point - a range of data should be included in order to make conclusions

  • And this site compares the openness of government data by country (2017 data) https://opendatabarometer.org/?_year=2017&indicator=ODB

  • Good point - knowledge of data reliability and quality is often really important - one way to help this is by providing rich metadata - which we talk about later in this week 2

  • Yes - sounds like the kind of thing the location app https://foursquare.com/ does

  • ECMWF (European Centre for Medium-range Weather Forecasts) has this free online module on data assimilation: https://www.ecmwf.int/assets/elearning/da/da1/story_html5.html

  • In week 3 we have at least three steps of interest for planning - one visualising urban benefits, one on land use (Thames estuary) derived from satellite data and another on spotting palm trees by satellites. Welcome to the course!

  • Good ideas - comms is an interesting addition.

  • Absolutely - good visualistaions reduce cognitive overload!

  • Thanks - it is a really good point on trying to summarise highlights out of large amounts of text is not something we have really dealt with here. I have a colleage who is currently facing this having issued a survey and is now trying to collate and make succinct sense of all the free text comments - and sometimes people do use word clouds for this - but any...

  • That's a really interesting point - including more of our senses than just visual. I had wondered about what the climate stripes (https://showyourstripes.info/) would sound like if each stripe was turned into tones - the rising pitch as it gets warmer over time.

  • Thanks for sharing the Johns Hopkins dashboard - it is an effective dashboard-style visualisation. I like their use of tabs to hide/display and therefore not overwhelm as well as the explicit inclusion of all the data sources.

  • Thanks - good suggestion - we're currently working on a project for diagnosing hot spots within urban areas using satellite data - with a view to assisting city planners on where things like green spaces might help

  • uncluttered, beautiful, informative

  • Thanks - that's a really good point - access to, investigating and valuing all go hand in hand.

  • That's facinating - important work. Thanks for the link.

  • The spheres are made by pufferfishdisplays.com

  • Great points - so much to consider - I have colleagues whose research is on making consistent datasets from different satellite missions over time - even as far back Nimbus from the 60s and 70s - article from NASA about these pioneering missons here
    https://www.nasa.gov/content/goddard/nimbus-nasa-remembers-first-earth-observations

  • Thanks for the comment - it is sometimes difficult to unpick data sources (we'll come onto metadata later this week).
    @JoEvans - I have some experience of MetOceanView staff (but not products) and they're a friendly bunch - it would be worth contacting them directly.

  • Welcome to week 2 - we'll talk open data, data science and metadata.

    One thing to highlight on open data is that some control can still be asserted on licensed resources e.g., the Creative Commons creativecommons.org/choose/ is an open source way to show the conditions under which your work can be used by others - which may be with no control (known as...

  • We touch very briefly this week on the process of data science - of which quality control and cleaning data is a standard thing to do - to remove e.g., corrupted data and how to account for missing data - depending on the datasets these processes and assumptions could impact any analysis and results.
    In 2010 physicists from Berkeley reanalysed global...

  • I had to look up World Clim - and that reminded me of something that we will touch on in week 2 - which is discovery of data. A group called re3 (registry of research data repositories) list reputable data sources worldwide - and this is the link for World Clim https://www.re3data.org/repository/r3d100011791

  • The principle of intelligent/useful filtering of data is a good point - we'll touch on that next week when talking about the challenges of data discovery.

  • Excellent idea!

  • Indeed - using evidence based policies, informed by environmental data is really important. And collecting the right data isn't always easy - one example is that often you have to collect data for a long time before it becomes really useful and reliable trends can be diagnosed.

  • Do check out our 'urban benefits to you' visualisation in week 3 - it analyses the kind of factors that you list e.g. green spaces

  • The interconnectedness of environmental data to so much of our lives!

  • Interesting thought - the benefits of data assimilation to gain the best view of a current spread of a disease (with imperfect data of course - which is also the case for weather) and then model forward... nice idea.

  • There are lots of examples in the course on the applications of big data to environmental science - enjoy the course!

  • You'll find that the course is about some of the underlying needs of big data - such as computing power, access to data and data science skills - along with examples on applications relevant to the full range of environmental data - from weather forecasting to ecology.

  • Agreed - a good way to think of big data - when it gets so large that you struggle to deal with it in the usual ways - it is difficult to store, processing takes a long time (hours or days instead of minutes) etc.

  • If you work in climate change you might be interested in this networking group on applying satellite data to the issue of climate (along with some broader topics) - sign up is free and it is a friendly group https://www.nceo.ac.uk/space4climate/

  • Welcome to the course! I'm finding more time to dip into the course that I might normally - and it is great to see all the discussions.

  • Good point - as environmental scientists we're all about the data and analysis, but - as we will touch on when talking about data science in week 2 and for visualisations in week 3 - providing clear information to others for decision-making is vital.

  • Welcome - this will be a nice overview for you of current topics in environmental science and how they are addressed through big data.

  • Welcome to the course! Most of our developers use Python and as an open source language there are several online courses for that - including on FutureLearn. R is also open source and widely used - often by those in environmental science who have a stats or biology or ecology interest.

  • Big Data, in my view, is often used and often not well defined. In week 1 we talk to Victoria at CEDA who deals with big data daily at one of the massive data centres, with dedicated computing power, for environmental science.

  • Welcome to the course - in weeks 1 and 2 we'll describe some of the issues, including computing needs and data science skills - then in week 3 we'll show some visualisations as lots of environmental data needs mapping and showing lots of different variables at once - along the way we'll talk citizen science too!

  • A great comment, depending on what you are trying to convey, simplicity can be more important and more powerful that being impressive or novel.

  • That is an amazing image, thanks!

  • In week 3 we show a visualisation tool we have developed for showing environmental factors in particular neighbourhoods - called BOUNTY

  • Big data is very useful for trends and patterns - of course one of the challenges is making sure there is a true signal to be found in the relationships - which is why sound statistical analysis is vital.

  • Good point. I used to work in issuing permits to extract water for irrigation which, due to scarcity of groundwater, meant that irrigation had to make sure it didn't over water - and keeping a close watch on the weather forecast and checking on soil moisture were both important for optimal efficiency.

  • Monitoring deforestation by satellites is currently done - in week 3 we'll even mention how individual trees can be counted from space.

  • Absolutely! Accounting for such variations across small distances in both the horizontal and vertical is one of the reasons that increasing resolution of, for example weather models, has been and still is a huge research and operational effort.

  • An example of open data on air pollution is Europe's Copernicus Air Monitoring Service - more info here https://atmosphere.copernicus.eu/

  • Defining how big something needs to be to be 'big data' has definitely changed over time. Companies and institutions worldwide are currently making their high performance computing facilities available so that Covid-19 is less of a big data problem than it might otherwise be.

  • I hope that the course provides some glimpses of the wider world of environmental big data - welcome to the course!

  • Spatial analysis is crucial for much of environmental science - so many of the examples in week 3 are based on mapping data

  • We have a couple of examples of land use applications in week 3

  • Check out step 1.13 this week - satellite data is a key component...

  • Welcome to the course. We'll see in week 2 that there are lots of open access datasets for environmental data - perhaps they'll inspire your data science applications.

  • Your work sounds really interesting and I hope that the course gives an overview of other applications.

  • And do take a look in week 1 at the video from Alan - about dealing with the complexities of environmental data

  • I agree that the term 'big data' will vary depending on the community - what is big to one sector (files too large to handle or store) is not to another.

  • Do tell us more about the data they're collecting - is there a web link to the work?

  • Some kind of 'big data' loop!

  • Good point - some active contributions and some that just happen through normal activities...

  • I contribute to big data by putting videos on YouTube - for various learning projects I'm involved in.

  • The IEA has a wind resource analysis tool RE-SAT which we mention briefly in the course and there's some more here https://www.re-sat.com/

  • We'll talk about citizen science contributions for the biological records centre with Tom August in week 3

  • Yes - as we'll come onto by week 3 - each action could be small, but the total will be big

  • We'll mention smart meters in week 2

  • Good comment - any one run may be a few data points, but the total of all those routes over time is big

  • Very interesting idea to move to carbon as the metric!

  • Welcome to week 2! Here we'll talk about open data and data science - along with the challenge of finding the right data in the first place. Here's the UK government website for open datasets https://data.gov.uk/

  • Indeed, sensitivity analysis of the data, to determine whether different groupings lead to different overall results - disambiguation.

  • Thanks for all the links!

  • The addition of 'Value' as the 5th V is interesting - I suppose that one person may find value in a dataset and others not - and perhaps value is a variant on 'Veracity'?

  • We talk about #ShowYourStripes in week 3

  • Thanks for sharing the link - the animation of annual ice from 1984 to 2016 is very absorbing and the page has a great explanation too.

  • Thanks for sharing the animation - a clear comparison over time of the change in the forest and using satellite data to do so.

  • Using cameras for monitoring in remote or hostile environments is a good point - just like satellite data, providing excellent coverage in time and space. We talk about citizen science in week 3!

  • Using mobile measurements is an interesting idea!

    Across the world there are lots of static ground level measurements continually being made for standard air pollutants (particulates, sulphur dioxide, nitrogen dioxide, carbon monoxide and others). But also there are baseline air pollution stations, such as this one in Tasmania, Australia, which monitor...

  • Personalised warnings sounds like it really has potential - especially if automated.
    A colleague of mine at the Met Office was involved in warnings for those with COPD (Chronic Obstructive Pulmonary Disease) - here's some details https://blog.metoffice.gov.uk/tag/chronic-obstructive-pulmonary-disease/

  • I also work on data ethics and completely agree that licensing and informed consent are vital. Most learned societies have codes of conduct for their members, tackling the issue at the personal level. Periodically there are calls for a Hippocratic oath for scientists - such as this from Hannah Fry earlier this year...

  • We'll see throughout the course that small data also has it's part to play - in fact week 3 is devoted to this topic

  • We also on a European open data portal for environmental datasets, including satellite data - check out NextGEOSS.eu

  • I like the expression 'endless swirl of information' - which is true of environmental data - the datasets keep growing every day as more data is collected - and recovering old datasets previously recorded on paper is also an active area of research and citizen science which we will cover in week 3.

  • We have Alan speaking about the business applications of environmental analytics later this week and, for infographics, we have one on metadata in week 2. Welcome to the course!

  • Really good point - having big data is one thing, but knowing what you want to do with it is important to guide your analysis and which other datasets you might want to use alongside.

  • A lot of the examples we present here do rely on spatial presentation - from methane detected by satellites to land use - check them out later in the course.

  • Permissions and licensing are a really good point - we don't cover these issues in depth, but we do talk about open data in week 2.

  • Welcome to the course! We're looking forward to sharing on big data for environmental applications - from the number crunching for weather forecasts to the skills needed for data science. We'll be here over the next three weeks to answer questions, read your facinating contributions and add any thoughts we have along the way. Tell us what you think and what...

  • Big data is a challenge, especially with the inherent uncertainties around measuring the environment and then turning those measurements into a mathematical model of a system - this will be mentioned through the course.

  • Welcome to the course! We have lots on the types of data used to analyse the environment, and in week 2 we have our data scientist, Ben, talk about how he approaches data analysis.

  • All those tiny actions adding up to big data!

  • Discovery is an interesting point, to find exactly what you want in a reasonable time is a significant challenge of big data. And thanks for mentioning weather forecasts... we'll mention computer models of the weather a few times through the course!

  • Thanks for mentioning qualitative research!

  • The yield of a crop is a good example, one number representing so many things - from exact seed type to time of planting to weather conditions and perhaps even harvesting method. And of course you'd need the metadata which told you the size of the field to be able to compare with others.

  • Agreed on time zone and what time means as an issue. Is it an instantaneous measurement at 11:00 or an average of the previous hour ending on that hour. For a meteorological wind measurement on a standard observation, it would most likely be taken in GMT and be the 10 min average ending at 11:00, and most likely measured in knots, but might be metres per...

  • In week 3, we have a nice feature on citizen science by Tom August and the Biological Records Centre - enjoy!

  • Thanks for joining the course. Happy for you to use the course resources for teaching such as video.

  • Good point that the 'big' of big data is the accumulation of bits and bytes of data.
    And to carry on your theme of palm trees... check out week 3, where Paula talks about counting palm trees from space...

  • @GearóidHolloway - we were having issues on Sunday, but now it's back and working