In many ways the challenge in communicating the complex and technical details behind Information Management has been my greatest joy over the past few years — and yet also frustrating. Standing before key decision makers in a number of organisations and trying to explain why this is so important, how it can radically change a business and what it could be can be incredibly exciting — when I get it right. Yet every time I walk away from a meeting feeling like I haven’t connected with a CEO or CTO, I feel like I’ve wasted an opportunity. Information Management and it’s related capabilities of analytics and business intelligence is critical in the Information Age and it is my driving desire to see all organisations introduced to the endless wonder of truly understanding how and why their business works. The only way I’ve found to do this is to place these complex and technical details into stories and analogies — examples if you will — of how and when this would be useful; taking my listeners and readers on a small journey until they see just how amazing this could all be.
With this in mind, in todays article I want to start a 3 part miniseries describing my efforts building a Smart House. Given I recently built a new place, this is one of those great moments when I get blog about something I’m actually doing right now! Along the way I hope to draw out some principles about Information Management and Data Analytics so that you can see parallels in how they might be useful in your own organisation.
ContextImagine I live in a city which is truly networked. Each house communicates with a city super computer which then uses this information to deliver power, water, gas and other services to my house to my house in the most efficient manner possible. The benefits are enormous — infrastructure costs are down, the power grid is stable and the city has seen an absolute boom in house prices!
To enable this incredible efficiency, the city mandates that owners are responsible for capturing data from their homes and deliver it to the network through a list of approved connections. Owners can do whatever they like within the bounds of their own home (provided it’s safe) and be as creative and interesting as they choose.
Because I’m a bit of a tech nerd (this is NOT mythical) and I love beautiful spaces, I’ve decided that I want to set up my house with some beautiful lighting to highlight key features, my artwork and provide a deep, warm ambiance. Furthermore, I want to contribute to reducing my energy footprint, have a limited budget and want a system which is simple and easy to use. In the future I want to look at storing all my usage data so that I can use some Artificial Intelligence methods to further improve my consumption and make it more efficient.
Step One: Building My Infrastructure
To start with, I sat down and considered what I wanted to do, reducing it down into four main goals:
Then I had to figure out how to make it all together. Ask myself — how can I get all these different things to work together? How can I meet all things important to me without spending months getting there? Luckily, because I was building a new house, it was pretty easy for me to build out a network which was easily extendable, able to connect to all different kinds of devices/things and be gathered together in a single point. For a small fee, I arranged with my builder to include fibre optic cabling through my house, including access points in the lounge, both bedrooms and office. This allowed me to plug in many tools, was easy to use, and provides ready access for guests and visitors. Furthermore, anything I added to my infrastructure would be modular and easy to upgrade.
The AnalogyFor many organisations, information infrastructure investment is one of the most difficult and expensive processes they are faced within the initial stages of building an effective information system. It requires an intimate knowledge of who you are as an organisation, where you want to go and a lot of smarts to design a network not just now, but for the future. I was fortunate — I knew exactly what I needed, had ultimate flexibility in getting there, and could afford to execute quickly on my plan. Many other organisations do not have this.
Yet I would also propose that this is one of the first and best times for an organisation to make a statement about who they are as a company, where they are headed and why this is important. Truly going through an infrastructure program requires many questions to be asked and answered, and is the opportunity of a lifetime to bring all the decision makers together to work it out.
The discussion about infrastructure investment should feed directly into your vision for YOUR organisation. It will establish what you need, how it’s to be delivered and how it will contribute to your business being amazing!I would also state that this is the best time to start building your Information Team. Giving them significant input into your plan can have huge benefits and save you a ton of cash and time!
Step Two: Choosing My Tools
Having built my infrastructure, the next challenge for me was to figure out what tools I wanted to use to achieve my vision. As it turns out, this is really an exercise in design thinking — something I deeply love. I already knew that I wanted to create something simple, beautiful and elegant, but now I had to work out exactly what that meant.
Because I feel that ambience is a way to make a room feel warm and inviting, I started by looking at lighting. It turns out there’s quite a number of players in the smart lighting space, all with varying degrees of quality, usefulness and connectivity. Fortunately, every single option was an LED of some description which is great — low power, long life and generally robust. After a bit of research, I decided to go with the Philips Hue option — it seemed to be well thought out, quite optionable and I really liked how easy it was to connect. It was a bit expensive, but for me the simplicity outweighed this; particularly the number of reviews which talked about how consistent it was. As an added bonus, the Hue range also included a throw light (Hue Bloom) and LED strips (LightStrips), both of which would be amazing ways to highlight things around my home.
The next challenge for me was working through the storage/routing tools. This is a dual challenge, as there are many different ways of connecting up routers and storage spaces. For me, because I pretty much run Apple products, I chose to go with an Apple Airport Time Capsule. This integrated seamlessly with my other Apple products and was easily plugged into my network. For my router, I chose a FritzBox wireless router. Again, it is easily configurable and really powerful, making it easy for me to connect multiple devices.
I want to make a quick note here about my routing decision. As you can see in the picture below, my earlier decision to wire everything together is now paying off. It is super simple to connect my router to all my fibre cables, and this massively extends my network. This is just one of those things where having spent some time in the infrastructure phase, I’m able to considerably speed up my toolsets space.
It all seemed to be going well, and there’s many parallels I could bring out of this story. Things like taking the time to work through the problem. Investing heavily in solutions which suit your values (i.e. simplicity for me) and not being afraid to reuse tech you already have — provided it makes sense.
Yet the final part of my initial build was where it all started to come unstuck. Powerpoints. Who would have thought that this was where it would all start to get difficult! Yet there I was trying to work through my values, figure out what to do and how to do it when I realised I was approaching the problem all wrong!
I’ll share the challenge and the solution next post, so stay tuned to see what happens :) As always, I’d love to hear and receive your feedback, thoughts, ideas and comments.
As I go around different organisations looking at their information systems and how they manage, analyse and use their data, there are a number of common themes which breed success. In some of my previous posts I’ve talked about how important it is to build a data driven culture; to employ the right people; and how important it is to build analytics and business intelligence upon a solid information management platform. All of these things are true — and in many cases not that much different from success in other areas of business. But today I want to share some of my thoughts and observations about the beauty of an amazing User Exerience and how a well designed and constructed user experience and user interface can increase information quality and convey decision information.
Improving Information Quality
I’ll start with what is arguably the most important aspect of information management — ensuring the accuracy of information. Accuracy of information is critical for any information system, as without trust in the information being provided it is impossible for analytics and ultimately business intelligence to succeed.
While a huge body of work is devoted to using machine learning and statistical methods to reduce the impact of incorrect information from users, one of the most surprising things I’ve observed about well designed systems is their use of visual cues to users to reduce user error. This isn’t just a minor observation — in an article by Oleysa Krysynyak entitled ‘How to Save Lives by Reducing User Interface Errors’, links are made between UI design and providing fast, reliable and easily understood information to doctors, literally saving lives!
Visual cues provide almost unnoticed pathways for users to enter information. Things like consistent color schemes (i.e. red is always bad, green is always good), consistent transactional interactions and a commitment to a reduction in clutter all contribute to providing a seamless experience for users and induce far less errors. As a result computing resources are freed up and the analytics and business intelligence which supports decision making increases in impact.
This is a non-trivial challenge. In teams I have worked with in the past, up to 50% of our time was spent working through each user interaction and relating it back to common themes we wanted to convey. Often times we would find that much of our programming construct needed to be changed to simplify variables and data tables, further refining our data sets and assumptions about users. In turn, this would create further discussion about how we wanted to convey information. Yet by sweating each detail of the user experience, and bringing an unrelenting commitment to convey information clearly to users, we would come up with sometimes amazingly simple answers to complex problems. The beauty of this kind of user experience is that the quality of the information being gathered is increased without the user even being aware of what is happening!
Decision Quality Impact
My second observation about well designed systems is their ability to convey often complex and comprehensive data to users in a manner they can understand and react too.
For me, this was bought home on a system I worked on which displayed some performance metrics for a platform. A previous organisation had come in and could track all sorts of interesting things — the uptime of each platform, the performance of different components for each platform and how frequently this had contributed to success. Yet as my team and I started to work through the consultation process with users, we quickly discovered that the presentation of this data wasn’t helping users at all. What they really wanted to know was which components failed the most, which components contributed the most to cost, and which stuff was going to be the most critical for them to solve. All of this information (other than the finance data) had already been gathered by the previous organisation, but it wasn’t helping the users of the system. As a result, the quality of their decision making hadn’t improved.
In contrast, great information systems come up with innovative ways to take a broad series of data sets, combine them together and present the results to users. In many cases, these teams behind these information systems end up finding solutions which are so simple in their elegance, and so well presented in the User Interface that their end users are entirely abstracted from the complexity of the problem!
This kind of User Experience is beautiful in its ability to allow individuals to focus on their own areas of expertise. Programs like Tableau and Highchartsallow User Experience designers to present complex information sets in a simple way. In turn, this allows users to use their expertise to contribute to the organisation. This in turn allows managers to know what is important and so on. The end result? Users are able to make accurate, timely decisions from the data being presented.
Users Don’t Hate the IT System
My final point in this post is one of my pet passions — building systems which actually help users. Far too often I see organisations which spend millions on marketing, talk a great game to customers and then punish their employees with substandard DOS based, text driven systems which require an intimate knowledge of search strings and building SQL queries to actually get information out of it.
These kind of back end systems don’t help users or customers at all, and almost always the outcome is counterproductive to the organisations goals. If users of the system hate it, they won’t use it to get good information, won’t be able to help their customers and they will figure out ways around it. As a result, the analytics of the information system reduce in quality, the organisation cannot get good business intelligence and everyone loses. All because someone somewhere didn’t take the time to get it right.
In contrast, great information systems become a tool to help users access the information they need to do what they’re doing better, faster, easier. When users don’t hate their IT system, everyone wins and a positive feedback loop is introduced. It’s definitely worth it!
So What To Do Now?
In my opinion, building a simple, elegant User Interface and User Experience is one of the great contributers to Information System success. I am constantly challenged in my work to keep refining systems, and often this refining starts with looking at how the user is presented, conveyed and interacts with data. It’s a time consuming process and while many people try to short cut the process, I’ve never found a way to do so without compromising effectiveness. For me there is no substitute for reading about psychology, learning to really listen to users and gaining an intimate knowledge of the goals of an organisation and what matters to them. I would challenge those who are reading this article to take the time to get this right. Take the time to support those UX designers in your team who really do sweat the detail, or, if you are one of those designers, I would challenge you take the time to keep investing in this area. In doing so, you’ll build information systems which will transform organisations into in the Information Age!!!
I hope this article has helped, and as always, please feel free to contact me on LinkedIn, Facebook, Twitter or in the comments below.
The Great User Challenge
You’ve drunk the Cool Aid, caught the dream, understand the vision. After many months of consideration and research, brainstorms and meetings with stakeholders, you’re ready to launch into analytics and bring your organisation into the Information Age. You know it’s going to take time, but you’ve invested in your leadership team and are building a data driven culture, backed by a cohesive data driven team. Funds are set aside for the change, and you’ve now become one of the agents for positive change in a data driven world.
So now you ask: Where to start?
Todays post is about answering this question and I’m going to look at three areas:
The User Concept
As always, the start of the answer is a discussion about the concept of Information Management and Analytics, both of which feed Business Intelligence.
Conceptually, Information Management is about people. In fact, if you abstract it a bit further, information itself is about people. From the perspective of business, all information is gathered, analysed and distributed to increase the efficiency and effectiveness of business. From this perspective, the discussion quickly becomes about how a user can measurably benefit an organisation, rather than being a problem which needs to be solved.
To further distil this concept, I offer this definition of a user:
Any person who regularly interacts with an information system.
There is no distinction here about internal and external users, nor is there any reference to automated trawlers of an information system (also known as bots). This is deliberate as an information system should to be built to provide a seamless connection between customers, clients, employees and managers. When any of these groups (or malicious users) choose to use automated methods to trawl information, the information system should be capable of handling this.
For many organisations, this concept is not new. It’s been used successfully in customer service, product selection and is a straightforward application of the laws of supply and demand. However, the application of this concept to information systems is almost unheard off.
Consider this: Almost every organisation in the world (outside of tech paradises like Silicon Valley and New York) uses Microsoft Word, Excel and Powerpoint to store, write and analyse information in various formats. Even when an organisation uses a backend system like SAP to start combining their information stores together, often times this is copied and pasted out of the ‘Datamart’, transferred into one of these applications and then shared via email.
Take a few moments to consider the time inefficiencies this introduces to workflow. Each point of interaction introduces delays, replication and error, all of which are considered necessary by the users of the system to do their job. As multiple departments get involved, the situation continues to get worse. Before you know it, people are creating spreadsheets to store their own little information empires (I call them spreadmarts), introducing macros…the list goes on.
The User Framework
In contrast today, I present a more excellent way. I call it ‘The User Framework’.
The User Framework is based around three assumptions:
There’s some interesting points to be drawn from this:
Firstly, any information system which uses this framework needs extra storage capacity. This is not actually a problem, and I’ll cover why in the technical part of this post.
Secondly, such an information system will change over time. Tracking what a user does and then analysing this to find efficiencies can provide some pretty radical results. For instance, when you find that a user is accessing a particular information daily, you can drastically increase efficiency by surfacing this information sooner, reducing search time. This is not to mention the efficiencies in being able to preprocess results.
Thirdly, this kind of information system creates an iterative improvement framework, where it gets better and better over time. This is important as it allows an organisations competitive advantage continue to improve.
This kind of system is incredibly empowering and enabling for organisations. Over time and with the right team, the quality of data will improve as will the analytics being performed. Furthermore, as the analytics team continues to go deeper into process flows and outcomes, the requirement for information will be refined, leading to a higher quality of business intelligence.
A post like this would not be complete without some mention of the technical aspects of such a proposal. I will state up front that I receive no commission from these mentions; these are just tools I have used and/or know and am impressed with.
In mentioning these things, I want to be clear that each organisation will take a different path to Information Management. I’ve worked with companies who outsource almost all of this stuff, through to ones who choose to do it entirely in house. I have also personally worked on building aspects of these systems, and it takes a while. As such, the mix of products you choose to do these things will be individual - however, I also believe the products mentioned below will meet almost every organisations needs.
I have also witnessed some pretty poor advice being given by software/hardware vendors in the past. Any organisation or IT staff who claims that increasing storage capacity or server power is difficult is not telling the truth. If you get this, find a good company to help you (like mine)!!!
Storage - Red Hat
This is becoming more and more of an issue. Many organisations are stuck using proprietary systems which charge astronomical amounts of money for adding storage and server capability. In my honest opinion, a better way forward is to look into the many products Red Hat offers. Not only is their code open sourced, they have a long history of figuring out ways to take legacy programs and import them with no loss of data.
Web Backend - Django or Ruby on Rails
Most organisations around the world are starting to realise that the most efficient way to create this kind of information system is using an internal web server of some kind. Both Django and Ruby on Rails are open source with massive communities.
Analytics Language - Python
There’s a lot of debate about this in the analytics world. My personal preference is Python, simply because its so broad. Furthermore, Python integrates with Red Hat and Django seamlessly, which for an end to end solution is pretty powerful.
Server Stuff - Red Hat Linux
As the concept of becoming data driven embeds more and more into business operations, and more and more information systems are brought online, one of the central questions which is asked by CEO’s and executives across the world is - ‘What kind of person, team or contractor should we employ to achieve what we need?’
Getting the right people involved in data driven business is the primary issue of the decade, with businesses rising and falling on the quality, diversity and integrity of those they entrust with their data.
In this post I want to share my observations on what I believe to be the top five defining characteristics of excellent Data Driven Teams. Teams of this nature create enormous competitive advantage for their organisations, finding new ways to increase efficiency and delivering decision quality information to decision makers. As a side benefit, they are often able to help businesses execute quickly on new market opportunities, creating a positive feedback cycle for all involved.
Getting the right people involved in data driven business is the primary issue of the decade!
#1 Passionate About the Business
Every single successful analytics team I’ve ever met has exhibited this characteristic. It’s not just about the way they dress or talk, or the lunches they have or perks they enjoy - there’s a buzz around them. When you hear them talk about their company and the intimate details about how the organisation works and how they found this AMAZING insight when they combined some different data sources, you walk away feeling inspired.
A great example of this is a post I recently read by the Air BnB team, talking about migrating their data. I’m not going to expand on the details of what they did (although it was impressive) but what I noticed is how passionate they were about their organisation. I could feel the excitement of the challenge oozing out of them, leaping out from the page. Yes, there was some technical details in the post, and yes, the challenge they faced was significant, but it was clear that they loved what they were doing and were determined to succeed.
This kind of passion is what separates organisations who do data because they’ve been told what a great idea it is from those organisations which see the transformational benefits it brings. For me, I was so inspired by the Air BnB post I actually applied for a job there!
#2 Know technology
The second defining characteristic of an excellent Data Driven Team is a fundamental commitment to becoming technical experts in their field. This is critical.
The Information Age is fast gathering pace, and every day there are new methods being released to market. Even over the past couple of years there has been the rise of No SQL database infrastructure, the introduction of Meteor, a continuing development in the thoughts and principles of REST based web access, big data analytics like Hadoop and so on. This is not to mention the incredible developments in cybersecurity including user behaviour analysis, encryption and the growing rise of mobile interaction.
For many organisations this constant barrage of technologies appears to be simply a very expensive and continuing hobby for a group of people who like doing interesting things with computers. In truth, each of these new technologies, architectures, ideas and concepts come with their own advantages and disadvantages and an excellent Data Driven Team will be able to explain this.
As time goes on, this defining characteristic will be clearly felt across the organisation as information becomes more timely, more accurate and more in depth. It will revolutionise the way you see data and create depth to your activity.
#3 Exercise Leadership at All Levels
Leadership is defined by John C Maxwell as
Leadership is not about titles, positions or flowcharts. It is about one life influencing another.
It is this ability to influence those around which is a defining characteristic of excellent Data Driven Teams. As organisations build effective informations systems and begin to dive deep into what it’s telling them, there comes a point where the data needs to be translated from geek language into decision maker language. Only excellent data analysts and scientists can make this link, figuring out the most effective graphs, plots and emails to send to the right people.
It goes deeper than this though. Understanding the right data almost invariably leads to tradeoffs between efficiency and effectiveness, often related to timeliness. Due to the nature of data analytics, these tradeoffs typically exist in the realm of extremely technical details, which means that only technical experts can really make a decision.
When this defining characteristic is missing, teams become paralysed by inaction and unable to present their case to the organisation for further action. The impact of this will not be felt immediately, but over time information quality will diminish and the usefulness of the analytics team to building competitive advantage will disappear.
#4 Able to Focus on the Big Picture
Many IT teams attempt to solve every individual issue on a case by case basis, never taking the time to look at the broader issues being raised. As a result, the teams end up with an increasing burden of supportability for individual solutions, effectively parcelling out their time on issues which of only minimal benefit to the broader organisations.
In contrast, one of the defining characteristics of excellent Data Driven Teams is the ability to contribute to the larger organisational goals. This means being able to assess the distinction between smaller issues with large implications and larger issues which have individual implications. Excellent teams often spend a lot of their time on whiteboards and in meeting rooms plotting data flows and implications, circling back to users and how to provide analytics - but always with the larger picture in mind. As a consequence the entire organisation benefits as information flows are streamlined and analytics steadily improved.
Ironically, when this is done correctly, the change within the organisation so subtle it’s often missed. Individuals end up happier and more motivated as they are being provided relevant data and decision makers at all levels feel more supported by the data they are being given - but often the actual Data Driven Teams fade into the background. Long term the organisation will streak ahead of the competition, creating an almost unbeatable competitive advantage.
#5 Team Players
Finally, excellent Data Driven Teams are defined by their ability to play as a team within a larger organisation. They are generally the first to get involved in a new idea or brainstorm, looking at ways to help, rather than hinder. They seek opportunities to get involved in departments and with staff so they can better understand requirements. And most importantly of all they realise that in many cases they are only one aspect of the broader organisation.
These kind of teams are an absolute pleasure to work with. They get the point. They want to help. They are incredibly invested in the organisation and they really want the company to succeed. For the organisation, it’s of huge benefit as they find new and better ways to improve information flows and analysis, providing a continuing competitive advantage. Returning to my example of the Air BnB data team, these kinds of teams are there to serve their organisation and take the abstract and mundane and make it amazingly relevant.
The end result of these defining characteristics are teams who provide incredible benefit to their organisation. They understand the business and care deeply about it, working hard help it succeed and grow. As new technologies come in they will continue to grow their organisations data driven capabilities and create enormous competitive advantage.
Take the time to invest in building your team, as they will become your single greatest advantage in a fast moving time in history!!!
As always, if you like what I'm saying add me on social media - I'd love to hear your comments!
As the Information Age continues to gather pace, there’s three terms which continue to pop up about becoming data driven:
These three areas are generally presented as three separate capabilities, each offering something the other areas can’t. This perception is reinforced as the companies offering these services continue to try to differentiate themselves by coming up with all sorts of creative names and descriptions. To make things even more confusing, there’s hundreds of different technical terms which get thrown in - things like Hadoop clusters, test driven development, responsive design and my favourite architecturally driven dissemination (whatever that means).
The result of all these words getting thrown around? Key decision makers who know the exact effect they want have no way of deciphering the noise to see if the method being proposed will give them this result.
In this post, I’m going to offer my perspective on these terms and explore the central question: Is this method going to give me the effect I need?
The Endstate: Competitive Advantage
It doesn’t really matter if you’re a not for profit, government organisation or good old profit driven enterprise, the entire purpose of organisations is to deliver lasting value to key stakeholders. It follows, then, that the entire purpose of becoming data driven is to allow your organisation to create a competitive advantage through the capture, analysis and dissemination of relevant data across your organisation. In my mind, I define this as:
To deliver decision quality information to decision makers when and how they need it.
And that’s where it really starts. There’s actually no point in investing in becoming data driven if there’s no competitive advantage to be gained. Furthermore, unless an organisation invests in becoming data driven with an understanding that this the greatest competitive advantage to be had, there will not be the will to go through what can often be an uncomfortable change process.
With this in mind, I present a picture on how these terms all relate:
Starting at the base of the triangle, I’ll tackle Information Management first. In a previous post, I go into detail about some of the aspects of what information management is, but in the context of this discussion, I’d like to add some thoughts.
Creating, controlling and delivering the flow of information across the organisation.
Put simply, Information Management is really the delivery, when and how mechanism of being data driven. There’s plenty of different terms and methods relating to this, and the companies selling this stuff will often talk at length about their Operating Systems, Hadoop clusters, storage solutions and why their technical solution is better than the oppositions. One day I'll write about some of my experience in this area, but it's key to understand that regardless of using Red Hat, Windows Server, SQL Storage or No SQL storage you're still ultimately building the pipes which connect an organisation.
It’s for this reason that Information Management is at the base of the triangle. The first challenge for any organisation is to figure out what on earth they’re trying to track (data), who needs what (information) and how to get it there (management). Understanding this flow and then enhancing it will literally change how you do business!!!
The natural desire once data is being gathered and information is being passed around, is to start analysing the day to day transactional nature of things to see what deeper insight can be gathered. There’s a fair bit of art and science which goes into this analysis, but really, from an effects perspective, analytics can be defined as:
Finding out what the gathered information is telling decision makers about their organisation.
This is where all the branches of analytics spring from. All the different methods which get mentioned in sales pitches are really about trying to give decision makers quality information about what is happening, relieving them and their staff of the burden of gathering this information themselves and it’s something I’ve written about in previous posts.
However there’s some pretty important intricacies about analytics which often get missed.
Firstly, analytics requires a pretty equal split between data scientists and operationally focused individuals. It’s imperative that any solution is developed in tandem with an organisations culture and decision making process.
Secondly, analytics is never ending (at this stage). It seems that the deeper you go into analytics, the more you realise that you’re just scratching the surface. For instance, in my latest work, I and a team of five individuals are tracking user interactions with data to start speeding up how quickly this information gets given to them. I’ll go into this in a later post, but in this instance, my team and I are actually developing analytics upon analytics, with predicted massive increases in efficiency. The take away point about this is there’s no such thing as a one off purchase in this area.
As the top part of the triangle, business intelligence spans the point at which analytics is fed through to decision makers. It requires an effective information system combined with quality analytics and will take a while to get going.
The art and science of providing decision quality information to all decision makers.
Theres a few points I’d like to make here about business intelligence.
Does this method give the effect required?
Hopefully by now a method of assessing how effectively you manage your data is being drawn out, along with a framework for building your own capabilities. Here’s a few questions I typically work through with any team I’m involved with to continue to build this framework.
To wrap up today, I’d like to take the time to encourage you if you’re on this journey. The Information Age transformation has just begun and it’s only gathering pace. Mastering the three areas discussed in this article will take time and be a hard journey, but it’s worth it when you start to see what can be discovered about your organisation!
The definition of an Epic Business
Epic businesses create lasting value for owners (shareholders), influencing their employees, communities and environment for good. It’s a huge challenge yet endlessly rewarding - something truly worth living for!!!
There’s a whole ton of stuff which goes into creating epic business. It starts with leadership (favourite author: John C Maxwell) and flows right into ethics, purpose, vision and process. Once you hit the practical side of things, you need excellent capital allocation, worthwhile returns on capital and ways of tracking relevant performance metrics - all supported by a management team who can create a highly motivated, passionate workforce.
Todays post is my argument that without epic analytics it’s impossible to build epic business!!!
Epic performance metrics
I’ll start with everyones favourite part of work: performance measurement - aka KPI’s, Bonus Indicators, What My Boss Rakes Me Over the Coals With or All Management Really Cares About.
There’s a bunch of research about how important tracking performance metrics is. Many management courses teach the theory and meaning of performance metrics, but the reality is getting this right creates incentive for people to come to work and means that everyone is pulling in the same direction. One leader of an epic company had this to say:
As you lead a group of people, you have an obligation to let them know where they stand.
Jack Welsh - CEO General Electric
Here’s the thing: Only an epic analytics team can make this possible!!!
As the Information Age gathers momentum, the speed at which business is conducted, combined with the nature of decision points being tracked results in enormous quantities of data pouring into information systems. All this data has to be analysed, considered, tested and turned into information - and getting this right is a combination of art, science and a touch of intuitive magic.
An epic analytics team gets this. They get the business, get the leaders intent and know how to take this river of data and transform it into a stream of useful information. Working with the IM team, they will build an analytics capability which will provide the kind of metrics most people can only dream about. All of sudden a business transforms into being data driven, changing everything!!!
Epic capital allocation
Within the world of value investing, championed by the legendary Warren Buffett, one of the greatest ways to create value for business owners (shareholders) is through effective capital allocation.
Capital allocation refers to the manner in which capital is invested on behalf of a company to build margins. The margins part is critical, because capital allocation is not just about increasing revenue - it can be as much about reducing costs or figuring out how improve processes as anything else.
Epic analytics in this area is crucial to creating epic businesses. One of the most profit destroying issues within organisations are the hidden costs of doing business, and it takes epic data analysis to figure out where and what they are. Once these hidden costs have been discovered, capital can be invested to eliminate this waste, creating value for owners. This process can take months or years and uncover all sorts of challenging issues - but the result is transformative. Capital allocation becomes exponentially more effective, with the flow on effect of freeing up more and more capital for other projects.
Epic employee motivation
Pretty much every leadership book ever has one point - motivating those around to achieve the goal. Epic CEO’s and founders are renowned for being able to motivate individuals and create motivating work spaces.
It’s interesting then to note that there’s a huge amount of research which shows that employees are most motivated when there is a clear link between an organisations values and their own - known as value driven organisations. It's even more interesting to note that one of the factors which comes up when this is discussed is employees desires that their performance be tracked and results mapped out - for instance in this article 'Values Driven Performance: Seven Strategies for Delivering Profit With Principles'
So here’s the thing - linking values to outcomes requires understanding what indicators drive these values and which measures of success need to be tracked. Planning on positively impacting a community for the long run? Define positive. Show employees what that means. Want to create products which change the world for good? Show me what you need to achieve to get there, per person, per day.
These are the kind of questions epic data analysis provides answers too. And if this capability is missing, it’s going to be impossible to get there!
And so it is that I reach the point of this post. Epic business requires epic data analytics.
Data analytics forms the glue which binds organisations together to pull in one direction. It dives into the data and begins to tease apart the questions of why and how and who and when and what. It gets involved in an organisation and brings insight, understanding and resolution. Data analytics allows you and every person who works for you to understand the same story and reasons and work together to fix them - testing, assessing and adjusting each step of the way as you see the results unfold.
For this reason, I submit to you - reader, CEO, CFO, CIO that you need data analytics.
It's your turn to host Christmas for your family this year. The tree is set, the fairy lights are glowing, the presents are sitting there tempting you with their promise of answered dreams and excitement. In your plan, you've decided that two days before Christmas is the time you'll be heading to the shops to buy all the fresh stuff you need for the big day (everyone knows that the day before Christmas is a total NIGHTMARE!). And so off you go to the shops armed with a giant shopping list of food, items and last minute panic purchases.
But wait - how did you generate this list? Where did it come from? Was it pure intuition, spur of the moment stuff - or was their a science or method behind it? What considerations did you make?
In today's post I'm going to deconstruct this process and explain how successful data analytics organisations work, (hopefully) drawing out some keys for you to use in your analytics journey. Read on to see if it helps!
#1 It's about people
My first point is the foundational point for successful analytics. People matter. Only in the context of helping, assisting and improving peoples lives is data given context and meaning.
In the example above, the primary driver is people. If you're anything like me, the first consideration would have been the number of people coming along, their likes and dislikes. This could have followed on to traditions within you or your partners family, and probably would have led on to questions about whether there were enough presents, Christmas crackers, drinks and so on. All of these requirements, can be traced back to one thing - the people.
In a similar manner, organisations who are successful in implementing data analytics start with people. Typically, they break this down into three major areas:
#2 It's about supporting decision making
My second point is that data analytics provides decision makers with information around their decision points - it does not take over the decision making process.
When it's done right, data analytics provides tools to build a picture of what is happening. In the above example, once you know the rough numbers of individuals coming, the next question to ask yourself is probably around what to eat. There's a lot of factors to consider in this - some of your guests might have specific allergies; or you may have some traditions which cannot be contravened. Each of these pieces of data will shape and guide what you purchase, but the point is that you remain in control of the process as the key decision maker.
Contrast this to a process where a computer program simply surveyed a mythical information system which had tracked everything all of your guests had eaten for their entire year, generating a list for you to buy. In this case, your system has staged a coup. You have lost control of the decision making process, generating results which lack context, theme or reason. Moreover, you'll probably get it wrong, because it's statistically unlikely that your guests eat a Christmas meal more often than 3-4 times a year.
In a similar way, the goal of data analytics and your data analytics team is provide accurate, relevant data around your decision making. Data analytics and your data analytics team are not there to replace your decision making process - simply to inform and decipher the data resident in your analytics system.
#3 It's about understanding why
The third thing I'd like to point out about successful data analytics is the continual pursuit of understanding WHY things occur.
This may seem like a pretty easy process, but it can get quite complicated very quickly. In the example above, lets take a look at why you made the decisions you did for food. Firstly, you surveyed the numbers attending, and came up with a figure you were fairly confident with - we would call this a confidence level. Then, you started to make decisions about what types of food to buy, and here is where it gets complex pretty quickly. Ask yourself 'why?'
To start with, you might consider your guests preferences, allergies, traditions and arrival times. This cuts away a list of things you definitely cannot cook or use. Then, diving down a bit further, you need to make an assessment of how much time you're going to be able contribute to this process (and the likelihood or otherwise of people assisting you). Going a bit deeper, you might start to consider when different foods are to be served, or what kind of serving capabilities you have on hand. Then, what sort of finances you are able to contribute. And the list goes on.
Your end game might be to serve a delicious meal and have a great time - but imagine this process repeated millions of times across all sorts of locations and figuring out ways to make this as efficient and effective (business words) as possible!!! That is what data analytics can do for you, but it means a singular commitment to the question WHY.
Why is consumption of these items at the level it is? Why do people keep making the same error on webpage when they do? Why does stuff break after this many uses? Why do I need new equipment? Don't be afraid of this process, rather embrace and pursue it and you'll be amazed at the transformation in your organisation!
#4 It's about the data
My final point in todays post comes back to my title - becoming data driven. Any information system, analytics capability and analytics team is only ever as good as the data being fed into it/them.
Successful organisations take time and care to invest in building systems which make it easy, simple and clear to their employees and customers on how to do this and do it well. This isn't just about how to guides and online videos; it's about your visual presentation, your design choices and your engagement with the process and team.
My starting example could be blown right out of the water if you had no idea who was coming, what they liked and what your traditions were. In this case, you would be presenting your assessment of a 'good' result, and while none of your guests would say so, you'd probably end up being 'that guy/girl' for the next few years!!!
Sadly, many companies today do this with their products and services, and as a result introduce risk and waste into what they're doing. Take the time to get it right and think about the people and I am confident that your organisation will prosper.
Hopefully this post has helped, and if so, feel free to drop me a line. I'm always looking to improve on my own knowledge and understanding, so if you feel I'm wrong, speak up! You can check out some of my other posts at http://www.creativeappnologies.com/data-driven-blog and I'm always looking for new challenges! Look forward to seeing you all next time,
As always, I'd love to hear your thoughts and comments. Use the tag #creativeappnologies to let me know.
So you've come this far. You've been reading about how amazing data analytics is, the power it has to transform your business and are thinking to yourself 'I'd like to get my organisation on board with this!'
Over the next few days you think about all the different bits and pieces of data you (might) need to understand. There's HEAPS. Then you brainstorm all the different databases your organisation has created, run, managed, used and accessed over the years. There's a TON. Then you try and think a little bit about how to manage this moving forward. It's a NIGHTMARE.
Before you know it the whole data analytics thing seems like it's just going to be too hard, too impossible, too big, too massive.
In todays blog post I want to have a look at this situation and break it down into some more manageable parts. Read on to find some assistance in your (apparent) nightmare.
The first key to an effective data analytics capability is capturing the experience and knowledge your organisation produces every day. This means your organisation needs to work towards delivering the right information, to the right people at the right time with as little hassle as possible.
These three rights and a no 'h' are really the key to information management. When they're are done correctly, your business will be transformed!!!
To get to this sweet spot of data transforming your company, it's likely that you'll find some pretty challenging issues to solve. Some of them will highly technical and involved, and some will be specific to your organisation, however there are several things I've observed really successful organisations do which always seem to work. Here they are:
Key #1 - Build a culture, not an event
This is perhaps the hardest challenge to overcome and yet in the end, it brings the greatest rewards.
When IM works properly, it captures what each individual does and shares it with as many people as possible without requiring duplication of effort. Put another way, effective IM declutters and simplifies work so individuals can do what they're good at: not waste time clicking through unnecessary IT steps or calling up tech support for (another) time wasting IT problem. This is a long term, ever improving culture, not a one off event!!!
When you create this kind of culture, you are prepositioning your organisation to succeed in data analytics. Individuals will start to see IT as a help, not a hinderance, and as a result, they begin to trust their IM team to assist them with issues.
Key #2 - Build a team, not a computer
Although Information Management is about data, it is the people who use and interact with it who really define the value of an effective IM system. As such, your IM team are critically important.
It is your IM team who will become the collective custodians of your data, and often times the success of your analytics will rest on their ability to access, manipulate and pass the right information to your data scientists. Furthermore, as your analytics capability grows, it will be your IM team who will pass the correct results back to the end users who need it. When this is done right, the culture of data driven business is reinforced and powerful results can be seen.
Key #3 - Start small, grow big
Building a culture and a team is really about expanding a circle of competence, and this starts by solving small problems and growing big. At the start of most organisational journeys there are a host of really critical issues to be solved. These issues may not be technically challenging, or as fun and exciting as analytics, but if they're not solved correctly, it can bring everything to a grinding halt!
As such, start with small areas of information management - solving problems for one team, or one area of the organisation. Take time to review the results (exhaustively) and be clear in feedback to the team about what is working and what is failing. Most importantly, take time to invest in solving the issues correctly.
Key #4 - Embrace Shadow IT
A recent survey found that more than half of CIO's fear shadow IT (this is where individuals within an organisation get frustrated with the current IT situation and go and start creating their own solutions). I remember reading once about a CIO who referred to shadow IT as the root of all the problems within his organisation.
This is interesting to me, because shadow IT represents an amazing opportunity for organisations to innovate and grow. It's quite straight forward to create an environment where people can experiment and play with organisational data in a safe and secure way - and really, if you're building a culture and team of effective IM, why not embrace it?
Key #5 - Resource IM properly
Finally, one of the greatest fears for organisations starting this journey is understanding the $ cost required to get them where they feel they need to go. Truth be told, this often quite a valid fear as the resources required to truly transform an IM system can be large.
Yet in another sense, the $ cost of such a transformation needs to be considered in the context of two factors: what the money is being spent on, and the cost of NOT spending this money, also known as the opportunity cost.
A transformational IM system can literally create efficiencies out of nothing. For example, recently I was asked to justify the cost of building a web-based data system, which my team and I had designed to be fully interactive. We were able to do so by showing that we could reduce the number of times a person had to 'click' for a simple task from 4 to 1 (75%). Using a conservative estimation of 3 seconds saved per click, per person, 30 times per hour, we calculated that we could save the organisation 56.25 minutes per week per person. There were 200 people in the organisation, which meant that we saved the organisation 187.5 hours per week. Then we showed how we could reduce the time taken to fill out forms through pre-filling from simple, contextual data base queries, saving approximately 100 hours per week. Finally, we showed a method of pre-alerting several parties separated by distance of upcoming issues, with a predicted saving of around $100,000 per month. In none of the cases had we required any change in organisational processes or new positions to be added - we had simply started to explore the power of an effective IM system. Yet in the context of savings, we generated several million dollars per month and transformed the ability of the organisation to meet it's needs.
In starting small and building big, and with the right support from the management of the organisation, the organisation is now starting to build a data analytics capability which I am heavily involved in. All from the start of changing how the information system worked.
What is Predictive Analytics?
Predictive analytics is a pretty amazing field of data science and has the potential to revolutionise the way we make decisions in business, so in this post I’d like to see if I can clarify just what the field of predictive analytics really is.
At its core…
Imagine for a moment you’re sitting at home planning your Christmas shopping. You’ve got a list of all the people you’d like to buy presents, and probably assigned each of them a rough dollar amount you’d like to spend on each. In your mind you’ve probably started working backwards from Christmas Day and figured out roughly what dates/times/places/websites you’d like to visit in order to find the perfect gift and then thought about how long it’ll take for those gifts to be found/packaged/wrapped/delivered so that they are ready for the big day. Without realising it, you have been performing many of the functions of predictive analytics.
Conceptually predictive analytics is precisely this: based on past experience, knowledge and insight, make predictions about a future event, including some kind of likelihood of this occurring.
As a business function…
It could be argued that businesses have been performing this function pretty much since the start of business. Each time a department store manager talks about getting ready for the holiday season, he or she is making a predictive statement based on past experience, knowledge and insight. I’m confident that the traders of Egypt and Babylon observed similar patterns in their own cultures and times and worked hard to maximise their returns based on this data. However, in the past 10 years or so, this kind of analysis has been put on steroids. These days its possible to not just predict trends accurately, but identify specific products to individuals, almost live. From a business perspective this is amazing! Simultaneously you can create efficiency, maximise sales and forecast far more accurately. There’s three components to this which I’ll expand on below.
It’s a matter of experience and knowledge…
The first component to predictive analytics is access to historical data. Put simply, the quantity, level of detail and accuracy of available data determines the accuracy of the predictions you might make. For instance, if you recorded every trip you made to a store for every day for 10 entire years, it’s pretty likely that you’d be able to predict quite accurately how much longer this trip might take over the Christmas period. Furthermore, you’d probably be able to make some observations about the general trend of these trips, the quality of the infrastructure and which set of traffic lights were the ‘worst’.
In a similar manner, the field of predictive analytics aims to relate previous data to future events. It does this through accessing as much data possible decision points or meaningful factors and then working out the trends, related outcomes and minute detail which the data indicates. This gives rise to the term ‘Big Data’ - a level and quantity of data which is so enormous it is literally beyond the human ability to understand, or put another way, the definition of ‘not being able to see the wood for the trees’. However the reason for this is pretty simple - the more data you have, the more accurate it is and longer it exists, the better results you will get.
Incidentally, when you read or hear about Amazon cloud technologies and the current race between Amazon Web Systems (https://aws.amazon.com), Microsofts Azure (https://azure.microsoft.com/en-us/?b=16.33) and Googles cloud technologies (https://cloud.google.com), you are hearing about the computing systems which make it possible to store, access and process these enormous data stores.
The second component to predictive analytics is the insight and it’s in this space that the true ‘magic’ of predictive analytics takes place. Insight is the ability to capture or make sense of the experience and knowledge gained previously. Returning to example of the trip to the shops, you gain insight when you take the knowledge and experience you’ve gained and apply it to your NEXT purchase. In a similar manner, the insight key of predictive analytics is the point where businesses can gain information to understand their choices.
A truly extensive analytics system can provide pretty amazing detail to businesses. Depending on the extent and range of your businesses you can start to understand all sorts of interesting things about what is happening behind the scenes of your organisation. This is especially true when elements of machine learning are introduced, as analytics systems systems can start to draw links between seemingly unrelated portions of businesses. In some reported examples, an analytics system was able to draw links between a supply chain issue and a particular factory producing substandard parts. In other cases, analytics systems has been used identify precise products to offer individuals, identify efficiencies in construction and whole range of interesting things.
In my mind, this is where predictive analytics is really heading. Assisting organisations in this field is pretty much my driving passion and I love the possibilities it opens up.
The final component to predictive analytics is what I tend to think of as the ‘useful’ part. Up until now, a predictive analytics system has been humming away, working through mountains of data and somehow making sense of it all. But in the predictive part two really important things happen.
Firstly, the system generates predictions. Put another way, this is where I get to see other people get excited about something that up until now they haven’t really understood. I’ll never forget the day I was able to show an engineering firm how to track a contract automatically, saving them millions of dollars in downtime and assisting their employees. The feel of excitement and relief was one of the highlights of my career so far!
Secondly, the system will generate some level of confidence for the predictions. This is a statistics term which recognises that the future is inherently uncertain and that there are ALWAYS outliers to a given range of results. There is always going to be that one customer who is unpredictable or changes his or her mind at the last minute. In most cases the level of confidence is set to such a level as to reduce any cost of incorrect predictions, however it is still something to keep in mind.
And so you have the basics of what predictive analytics is. I hope now you’re starting to see just how incredible this field and why it’s so interesting and exciting. I’ve got so much more to share on this topic so please feel free to drop me line or leave comments and I’ll respond when I can.
Some examples of predictive analytics at work…
Ubers redesign is really a UI function supporting predictive analytics
The cloud applications of Amazon, Azure and so on are really underpinning the massive amounts of data required to support predictive analytics
As a passionate data scientist, it is my honour and privilege to work with some of the most intelligent, interesting and driven individuals on the planet. As a result of my recent efforts within an organisation I was offered postgraduate study at the University of New South Wales focusing on predictive analytics across a logistics supply chain.
Yet despite the calibre of the individuals I work with and my general passion for this field of work, I remain often discouraged by a lack of understanding about the field of data analytics and the tremendous value it can add to companies. This blog is my personal effort to add my voice to this commentary. Hopefully I can be insightful and interesting, at times offering advice, training and direction to what is an incredibly interesting and diverse field.
I'd like to start by laying out what I think data analytics is and why it's important.
What is data analytics?
Data analytics is about taking mountains of data and analysing it to see what insights can be drawn gained through synthesis, in-depth knowledge and the art of extraction.
Synthesis. In a a modern logistics framework, there is literally hundreds of thousands of data points per item being recorded, everything from their GPS location, to check in times at key nodes. At a very basic level, this data is pretty useless and frankly annoying/overwhelming. I mean a piece of freight is going to arrive when it arrives right?
Synthesis is about taking these data points and starting to look at them in combination with lots of other disparate pieces of data to get insights. For instance, if your logistics network was global in nature, then you might want to look at normal weather patterns and understand the impact sunny days could have on your network and optimise around that. Or you might choose to look at your workforce and understand the impact substandard equipment has on your bottom line. Or you might want to look at the maintenance cycle of your assets and change it around to reduce total downtime. The point is that in each case, the insight was gained through combining different bits of information to produce a better understanding of the whole.
In-depth knowledge. Truly effective data analysis is a combination of synthesis combined with a thorough understanding of the subject being studied. For me this was shown in my MAGNUS project. The engineering firm I was working for had massive inefficiencies generated through a stove piped information environment, leading me to be confident in achieving a good result through simple synthesis. However, it was when I befriended one of the other engineers and was able to use his expertise in their engineering decision tracking tool that the savings ramped up into the millions of dollars. This was because it was in knowing and understanding the process that we were able to see the opportunity and make effective changes.
The Art of Extraction. For me this is the true test of effective data analytics. The process of synthesis and in-depth knowledge can produce many amazing insights into what is happening in an organisation. But sometimes those insights are pretty useless. True data analytics is just as much about filtering out the noise as delivering insights to end users.
So Why is this Important?
Data analytics has the potential to really revolutionise many organisations. It's very nature is about fully understanding and quantifying the risks, rewards and opportunities inherent in a choice. As a result, when data analytics is employed correctly it provides deep insight into the impact of decisions, individuals and equipment on the desired outcome.
A great recent example of this lies in Walmarts recent decision to continue investing in streamlining their trucks and delivery network. By analysing the routes they typically drove, they were able to increase the economy of their network by about 2% by reducing wind drag on their vehicle fleet. Furthermore, by identifying where the largest hold ups were in their network, they were able to redistribute deliveries and decrease overall down time.
For many organisations this kind of insight can improve returns dramatically. It increases competitive advantage and decreases risk.
I love analysing data. I've done it for nearly 10 years now in various shapes and forms, and for me it's an endless world of wonder. There's nothing else I'd rather be doing!