The unintended consequences of how the products we make are being used is becoming more important in the age of digital.
Just as some unintended consequences provide a positive benefit such as Twitter user Chris Messina suggesting that the # sign be used to designate groups on twitter, some product features may lead to devastating effects.
This post was inspired by the story that Channel 4 News in the UK has discovered that Amazon’s algorithm guides users to the necessary chemical combinations for producing explosives and incendiary devices.
Ingredients which are innocent on their own are suggested for purchase together as “Frequently bought together” products, as it does with all other goods.
With the Parson’s Green tube incident fresh in Londoner’s minds, It got me thinking – who in an organisation is responsible for the forward thinking to either plan against these negative consequences, or is tasked to regularly check that products and services are not being used in a negative way.
We could have a whole discussion around the positives and negatives of encryption, but the Amazon story is a reminder we need to need to be alert everywhere, not just in public.
