Where does all the data go?
Now that we’ve thought about what data consists of, we can now think about how we interact with data. As technology has advanced, processing data has become quicker and easier, but what does that mean for you?
One of the earliest forms of data storage is the tally stick. This was a piece of notched wood, split in half so two people could keep a record of a transaction. The earliest tally sticks are about 30,000 years old, which shows how long capturing and recording data has been important to people.
The arrival of new mathematical methods over the centuries made turning data into information more scientific, but it remained a manual process. In order to process data quickly and accurately, you needed machines.
The need for speed
The roots of the data processing machines we call computers are in the early 19th century invention of the punched card. Originally designed for cloth weaving, punched cards allowed data to be stored in a form that was easy for a machine to read. Inventor Charles Babbage realised punched cards could be used for more than just cloth and designed the Analytical Engine, the first modern programmable computer.
Modern electronic computing got underway in the 1940s, with computers being used to crack codes and process weapons data during the second world war. After the war, computers started to move into business. In 1951, British cafe company J Lyons built LEO I, the first office computer. LEO allowed the company to use data from orders in its hundreds of shops to calculate accurate production requirements, delivery schedules, invoices, and costings overnight.
LEO became the foundation of the next 20 years of so of data processing in business. Punched cards were slowly replaced by magnetic tape, and then floppy disks and hard drives. Computers got faster and could store more data, but how they were used remained fairly static, with large machines in air conditioned rooms tended by technicians. These were not that different to the server rooms and data centres used by Google, Microsoft and Amazon today to deliver everything from search results to a book for your birthday.
What has changed is speed. The machines we use to process data are orders of magnitude faster, but equally important is the speed of transmission of data. Data for LEO was not machine-readable. It was phoned in by humans to other humans, who then manually entered the data into the computer. Now, the equivalent would be a machine in a store talking directly to a machine in a data centre in milliseconds.
The advent of big data
The internet has revolutionised the collection and processing of data. It allows data to be stored in ‘the cloud’ in thousands of linked data centres across the world. The speed of the network is so fast that for most purposes it doesn’t matter if your data is in Docklands or Detroit. Data is collected by machines which live in our pockets - smartphones - transmitted to servers across the world, processed, and then fed back to us. A fraction of a second delay is noticeable to us, now.
Because data is so easy to collect, the amount which can be held is enormous. Vast sets of ‘big data’, let companies do things that have never been possible before. Google holds, on average, 5GB of data about you. That’s about the same amount of data as the text of 20,000 books.
Information has always been bought and sold, but data is now a commodity to be bought, sold and even hoarded to benefit those who own it.
What is your data worth? To a business, not much: the value of ‘big data’ is in aggregating large amounts of data to spot patterns at a global scale, not the value of a single individual. To you, though, that data is hugely valuable: to a company building a system to spot cancers from a huge set of medical records, one patient record is worth almost nothing. If you’re the patient, that record is worth your life.
Have your say:
You’ve seen in this article that as technology has advanced, it is now easier and faster than ever to gather and process data. What does this mean for our contemporary relationship with data and how it’s used?
Share your thoughts in the Comments.