It's your turn to host Christmas for your family this year. The tree is set, the fairy lights are glowing, the presents are sitting there tempting you with their promise of answered dreams and excitement. In your plan, you've decided that two days before Christmas is the time you'll be heading to the shops to buy all the fresh stuff you need for the big day (everyone knows that the day before Christmas is a total NIGHTMARE!). And so off you go to the shops armed with a giant shopping list of food, items and last minute panic purchases.
But wait - how did you generate this list? Where did it come from? Was it pure intuition, spur of the moment stuff - or was their a science or method behind it? What considerations did you make?
In today's post I'm going to deconstruct this process and explain how successful data analytics organisations work, (hopefully) drawing out some keys for you to use in your analytics journey. Read on to see if it helps!
#1 It's about people
My first point is the foundational point for successful analytics. People matter. Only in the context of helping, assisting and improving peoples lives is data given context and meaning.
In the example above, the primary driver is people. If you're anything like me, the first consideration would have been the number of people coming along, their likes and dislikes. This could have followed on to traditions within you or your partners family, and probably would have led on to questions about whether there were enough presents, Christmas crackers, drinks and so on. All of these requirements, can be traced back to one thing - the people.
In a similar manner, organisations who are successful in implementing data analytics start with people. Typically, they break this down into three major areas:
#2 It's about supporting decision making
My second point is that data analytics provides decision makers with information around their decision points - it does not take over the decision making process.
When it's done right, data analytics provides tools to build a picture of what is happening. In the above example, once you know the rough numbers of individuals coming, the next question to ask yourself is probably around what to eat. There's a lot of factors to consider in this - some of your guests might have specific allergies; or you may have some traditions which cannot be contravened. Each of these pieces of data will shape and guide what you purchase, but the point is that you remain in control of the process as the key decision maker.
Contrast this to a process where a computer program simply surveyed a mythical information system which had tracked everything all of your guests had eaten for their entire year, generating a list for you to buy. In this case, your system has staged a coup. You have lost control of the decision making process, generating results which lack context, theme or reason. Moreover, you'll probably get it wrong, because it's statistically unlikely that your guests eat a Christmas meal more often than 3-4 times a year.
In a similar way, the goal of data analytics and your data analytics team is provide accurate, relevant data around your decision making. Data analytics and your data analytics team are not there to replace your decision making process - simply to inform and decipher the data resident in your analytics system.
#3 It's about understanding why
The third thing I'd like to point out about successful data analytics is the continual pursuit of understanding WHY things occur.
This may seem like a pretty easy process, but it can get quite complicated very quickly. In the example above, lets take a look at why you made the decisions you did for food. Firstly, you surveyed the numbers attending, and came up with a figure you were fairly confident with - we would call this a confidence level. Then, you started to make decisions about what types of food to buy, and here is where it gets complex pretty quickly. Ask yourself 'why?'
To start with, you might consider your guests preferences, allergies, traditions and arrival times. This cuts away a list of things you definitely cannot cook or use. Then, diving down a bit further, you need to make an assessment of how much time you're going to be able contribute to this process (and the likelihood or otherwise of people assisting you). Going a bit deeper, you might start to consider when different foods are to be served, or what kind of serving capabilities you have on hand. Then, what sort of finances you are able to contribute. And the list goes on.
Your end game might be to serve a delicious meal and have a great time - but imagine this process repeated millions of times across all sorts of locations and figuring out ways to make this as efficient and effective (business words) as possible!!! That is what data analytics can do for you, but it means a singular commitment to the question WHY.
Why is consumption of these items at the level it is? Why do people keep making the same error on webpage when they do? Why does stuff break after this many uses? Why do I need new equipment? Don't be afraid of this process, rather embrace and pursue it and you'll be amazed at the transformation in your organisation!
#4 It's about the data
My final point in todays post comes back to my title - becoming data driven. Any information system, analytics capability and analytics team is only ever as good as the data being fed into it/them.
Successful organisations take time and care to invest in building systems which make it easy, simple and clear to their employees and customers on how to do this and do it well. This isn't just about how to guides and online videos; it's about your visual presentation, your design choices and your engagement with the process and team.
My starting example could be blown right out of the water if you had no idea who was coming, what they liked and what your traditions were. In this case, you would be presenting your assessment of a 'good' result, and while none of your guests would say so, you'd probably end up being 'that guy/girl' for the next few years!!!
Sadly, many companies today do this with their products and services, and as a result introduce risk and waste into what they're doing. Take the time to get it right and think about the people and I am confident that your organisation will prosper.
Hopefully this post has helped, and if so, feel free to drop me a line. I'm always looking to improve on my own knowledge and understanding, so if you feel I'm wrong, speak up! You can check out some of my other posts at http://www.creativeappnologies.com/data-driven-blog and I'm always looking for new challenges! Look forward to seeing you all next time,
As always, I'd love to hear your thoughts and comments. Use the tag #creativeappnologies to let me know.
So you've come this far. You've been reading about how amazing data analytics is, the power it has to transform your business and are thinking to yourself 'I'd like to get my organisation on board with this!'
Over the next few days you think about all the different bits and pieces of data you (might) need to understand. There's HEAPS. Then you brainstorm all the different databases your organisation has created, run, managed, used and accessed over the years. There's a TON. Then you try and think a little bit about how to manage this moving forward. It's a NIGHTMARE.
Before you know it the whole data analytics thing seems like it's just going to be too hard, too impossible, too big, too massive.
In todays blog post I want to have a look at this situation and break it down into some more manageable parts. Read on to find some assistance in your (apparent) nightmare.
The first key to an effective data analytics capability is capturing the experience and knowledge your organisation produces every day. This means your organisation needs to work towards delivering the right information, to the right people at the right time with as little hassle as possible.
These three rights and a no 'h' are really the key to information management. When they're are done correctly, your business will be transformed!!!
To get to this sweet spot of data transforming your company, it's likely that you'll find some pretty challenging issues to solve. Some of them will highly technical and involved, and some will be specific to your organisation, however there are several things I've observed really successful organisations do which always seem to work. Here they are:
Key #1 - Build a culture, not an event
This is perhaps the hardest challenge to overcome and yet in the end, it brings the greatest rewards.
When IM works properly, it captures what each individual does and shares it with as many people as possible without requiring duplication of effort. Put another way, effective IM declutters and simplifies work so individuals can do what they're good at: not waste time clicking through unnecessary IT steps or calling up tech support for (another) time wasting IT problem. This is a long term, ever improving culture, not a one off event!!!
When you create this kind of culture, you are prepositioning your organisation to succeed in data analytics. Individuals will start to see IT as a help, not a hinderance, and as a result, they begin to trust their IM team to assist them with issues.
Key #2 - Build a team, not a computer
Although Information Management is about data, it is the people who use and interact with it who really define the value of an effective IM system. As such, your IM team are critically important.
It is your IM team who will become the collective custodians of your data, and often times the success of your analytics will rest on their ability to access, manipulate and pass the right information to your data scientists. Furthermore, as your analytics capability grows, it will be your IM team who will pass the correct results back to the end users who need it. When this is done right, the culture of data driven business is reinforced and powerful results can be seen.
Key #3 - Start small, grow big
Building a culture and a team is really about expanding a circle of competence, and this starts by solving small problems and growing big. At the start of most organisational journeys there are a host of really critical issues to be solved. These issues may not be technically challenging, or as fun and exciting as analytics, but if they're not solved correctly, it can bring everything to a grinding halt!
As such, start with small areas of information management - solving problems for one team, or one area of the organisation. Take time to review the results (exhaustively) and be clear in feedback to the team about what is working and what is failing. Most importantly, take time to invest in solving the issues correctly.
Key #4 - Embrace Shadow IT
A recent survey found that more than half of CIO's fear shadow IT (this is where individuals within an organisation get frustrated with the current IT situation and go and start creating their own solutions). I remember reading once about a CIO who referred to shadow IT as the root of all the problems within his organisation.
This is interesting to me, because shadow IT represents an amazing opportunity for organisations to innovate and grow. It's quite straight forward to create an environment where people can experiment and play with organisational data in a safe and secure way - and really, if you're building a culture and team of effective IM, why not embrace it?
Key #5 - Resource IM properly
Finally, one of the greatest fears for organisations starting this journey is understanding the $ cost required to get them where they feel they need to go. Truth be told, this often quite a valid fear as the resources required to truly transform an IM system can be large.
Yet in another sense, the $ cost of such a transformation needs to be considered in the context of two factors: what the money is being spent on, and the cost of NOT spending this money, also known as the opportunity cost.
A transformational IM system can literally create efficiencies out of nothing. For example, recently I was asked to justify the cost of building a web-based data system, which my team and I had designed to be fully interactive. We were able to do so by showing that we could reduce the number of times a person had to 'click' for a simple task from 4 to 1 (75%). Using a conservative estimation of 3 seconds saved per click, per person, 30 times per hour, we calculated that we could save the organisation 56.25 minutes per week per person. There were 200 people in the organisation, which meant that we saved the organisation 187.5 hours per week. Then we showed how we could reduce the time taken to fill out forms through pre-filling from simple, contextual data base queries, saving approximately 100 hours per week. Finally, we showed a method of pre-alerting several parties separated by distance of upcoming issues, with a predicted saving of around $100,000 per month. In none of the cases had we required any change in organisational processes or new positions to be added - we had simply started to explore the power of an effective IM system. Yet in the context of savings, we generated several million dollars per month and transformed the ability of the organisation to meet it's needs.
In starting small and building big, and with the right support from the management of the organisation, the organisation is now starting to build a data analytics capability which I am heavily involved in. All from the start of changing how the information system worked.
What is Predictive Analytics?
Predictive analytics is a pretty amazing field of data science and has the potential to revolutionise the way we make decisions in business, so in this post I’d like to see if I can clarify just what the field of predictive analytics really is.
At its core…
Imagine for a moment you’re sitting at home planning your Christmas shopping. You’ve got a list of all the people you’d like to buy presents, and probably assigned each of them a rough dollar amount you’d like to spend on each. In your mind you’ve probably started working backwards from Christmas Day and figured out roughly what dates/times/places/websites you’d like to visit in order to find the perfect gift and then thought about how long it’ll take for those gifts to be found/packaged/wrapped/delivered so that they are ready for the big day. Without realising it, you have been performing many of the functions of predictive analytics.
Conceptually predictive analytics is precisely this: based on past experience, knowledge and insight, make predictions about a future event, including some kind of likelihood of this occurring.
As a business function…
It could be argued that businesses have been performing this function pretty much since the start of business. Each time a department store manager talks about getting ready for the holiday season, he or she is making a predictive statement based on past experience, knowledge and insight. I’m confident that the traders of Egypt and Babylon observed similar patterns in their own cultures and times and worked hard to maximise their returns based on this data. However, in the past 10 years or so, this kind of analysis has been put on steroids. These days its possible to not just predict trends accurately, but identify specific products to individuals, almost live. From a business perspective this is amazing! Simultaneously you can create efficiency, maximise sales and forecast far more accurately. There’s three components to this which I’ll expand on below.
It’s a matter of experience and knowledge…
The first component to predictive analytics is access to historical data. Put simply, the quantity, level of detail and accuracy of available data determines the accuracy of the predictions you might make. For instance, if you recorded every trip you made to a store for every day for 10 entire years, it’s pretty likely that you’d be able to predict quite accurately how much longer this trip might take over the Christmas period. Furthermore, you’d probably be able to make some observations about the general trend of these trips, the quality of the infrastructure and which set of traffic lights were the ‘worst’.
In a similar manner, the field of predictive analytics aims to relate previous data to future events. It does this through accessing as much data possible decision points or meaningful factors and then working out the trends, related outcomes and minute detail which the data indicates. This gives rise to the term ‘Big Data’ - a level and quantity of data which is so enormous it is literally beyond the human ability to understand, or put another way, the definition of ‘not being able to see the wood for the trees’. However the reason for this is pretty simple - the more data you have, the more accurate it is and longer it exists, the better results you will get.
Incidentally, when you read or hear about Amazon cloud technologies and the current race between Amazon Web Systems (https://aws.amazon.com), Microsofts Azure (https://azure.microsoft.com/en-us/?b=16.33) and Googles cloud technologies (https://cloud.google.com), you are hearing about the computing systems which make it possible to store, access and process these enormous data stores.
The second component to predictive analytics is the insight and it’s in this space that the true ‘magic’ of predictive analytics takes place. Insight is the ability to capture or make sense of the experience and knowledge gained previously. Returning to example of the trip to the shops, you gain insight when you take the knowledge and experience you’ve gained and apply it to your NEXT purchase. In a similar manner, the insight key of predictive analytics is the point where businesses can gain information to understand their choices.
A truly extensive analytics system can provide pretty amazing detail to businesses. Depending on the extent and range of your businesses you can start to understand all sorts of interesting things about what is happening behind the scenes of your organisation. This is especially true when elements of machine learning are introduced, as analytics systems systems can start to draw links between seemingly unrelated portions of businesses. In some reported examples, an analytics system was able to draw links between a supply chain issue and a particular factory producing substandard parts. In other cases, analytics systems has been used identify precise products to offer individuals, identify efficiencies in construction and whole range of interesting things.
In my mind, this is where predictive analytics is really heading. Assisting organisations in this field is pretty much my driving passion and I love the possibilities it opens up.
The final component to predictive analytics is what I tend to think of as the ‘useful’ part. Up until now, a predictive analytics system has been humming away, working through mountains of data and somehow making sense of it all. But in the predictive part two really important things happen.
Firstly, the system generates predictions. Put another way, this is where I get to see other people get excited about something that up until now they haven’t really understood. I’ll never forget the day I was able to show an engineering firm how to track a contract automatically, saving them millions of dollars in downtime and assisting their employees. The feel of excitement and relief was one of the highlights of my career so far!
Secondly, the system will generate some level of confidence for the predictions. This is a statistics term which recognises that the future is inherently uncertain and that there are ALWAYS outliers to a given range of results. There is always going to be that one customer who is unpredictable or changes his or her mind at the last minute. In most cases the level of confidence is set to such a level as to reduce any cost of incorrect predictions, however it is still something to keep in mind.
And so you have the basics of what predictive analytics is. I hope now you’re starting to see just how incredible this field and why it’s so interesting and exciting. I’ve got so much more to share on this topic so please feel free to drop me line or leave comments and I’ll respond when I can.
Some examples of predictive analytics at work…
Ubers redesign is really a UI function supporting predictive analytics
The cloud applications of Amazon, Azure and so on are really underpinning the massive amounts of data required to support predictive analytics
As a passionate data scientist, it is my honour and privilege to work with some of the most intelligent, interesting and driven individuals on the planet. As a result of my recent efforts within an organisation I was offered postgraduate study at the University of New South Wales focusing on predictive analytics across a logistics supply chain.
Yet despite the calibre of the individuals I work with and my general passion for this field of work, I remain often discouraged by a lack of understanding about the field of data analytics and the tremendous value it can add to companies. This blog is my personal effort to add my voice to this commentary. Hopefully I can be insightful and interesting, at times offering advice, training and direction to what is an incredibly interesting and diverse field.
I'd like to start by laying out what I think data analytics is and why it's important.
What is data analytics?
Data analytics is about taking mountains of data and analysing it to see what insights can be drawn gained through synthesis, in-depth knowledge and the art of extraction.
Synthesis. In a a modern logistics framework, there is literally hundreds of thousands of data points per item being recorded, everything from their GPS location, to check in times at key nodes. At a very basic level, this data is pretty useless and frankly annoying/overwhelming. I mean a piece of freight is going to arrive when it arrives right?
Synthesis is about taking these data points and starting to look at them in combination with lots of other disparate pieces of data to get insights. For instance, if your logistics network was global in nature, then you might want to look at normal weather patterns and understand the impact sunny days could have on your network and optimise around that. Or you might choose to look at your workforce and understand the impact substandard equipment has on your bottom line. Or you might want to look at the maintenance cycle of your assets and change it around to reduce total downtime. The point is that in each case, the insight was gained through combining different bits of information to produce a better understanding of the whole.
In-depth knowledge. Truly effective data analysis is a combination of synthesis combined with a thorough understanding of the subject being studied. For me this was shown in my MAGNUS project. The engineering firm I was working for had massive inefficiencies generated through a stove piped information environment, leading me to be confident in achieving a good result through simple synthesis. However, it was when I befriended one of the other engineers and was able to use his expertise in their engineering decision tracking tool that the savings ramped up into the millions of dollars. This was because it was in knowing and understanding the process that we were able to see the opportunity and make effective changes.
The Art of Extraction. For me this is the true test of effective data analytics. The process of synthesis and in-depth knowledge can produce many amazing insights into what is happening in an organisation. But sometimes those insights are pretty useless. True data analytics is just as much about filtering out the noise as delivering insights to end users.
So Why is this Important?
Data analytics has the potential to really revolutionise many organisations. It's very nature is about fully understanding and quantifying the risks, rewards and opportunities inherent in a choice. As a result, when data analytics is employed correctly it provides deep insight into the impact of decisions, individuals and equipment on the desired outcome.
A great recent example of this lies in Walmarts recent decision to continue investing in streamlining their trucks and delivery network. By analysing the routes they typically drove, they were able to increase the economy of their network by about 2% by reducing wind drag on their vehicle fleet. Furthermore, by identifying where the largest hold ups were in their network, they were able to redistribute deliveries and decrease overall down time.
For many organisations this kind of insight can improve returns dramatically. It increases competitive advantage and decreases risk.
I love analysing data. I've done it for nearly 10 years now in various shapes and forms, and for me it's an endless world of wonder. There's nothing else I'd rather be doing!