Sign in

I’m a professional data + product leader trained in comp + info science. I code, write, take photos for fun. https://wilsonwong.co

From time to time, we come up with ideas and speculate about how they might play out in reality. Take, for instance, a food scientist who believes that a new flavour enhancer could improve consumer demand, or a pharmaceutical company that have developed a medication which they believe could relieve high-blood pressure. At other times, we may have two or more variations of an idea and need to choose between them. Consider a chemist who need to find out which of his two new fertiliser formulations would deliver the best growth for nightshade plants compared to the prevailing one.

Controlled…

In a previous 3-part series, we discussed the main hurdles limiting the value that businesses get from data. In particular, we highlighted the importance of the timing of investment into establishing data infrastructure, recruiting data professionals, etc. The reasons are clear. Any significant returns on investment that you make on data are likely to come from their use in conjunction with AI techniques such as the various ML algorithms in your products. This is when your Data Science/ML teams come in, to explore the data, build the ML models, and package them up into what I call AI services that…

There has been a lot of talk in the recent years about the use of AI capabilities or machine-learned solutions to improve digital products or services and user experience. If we can cut through the hype and have the necessary building blocks in place as covered in Four Hurdles To Creating Value From Data, there is legitimate value to be had from the use of data and AI to make better products in many verticals. For tech companies that are founded on strong engineering practices and find themselves knee deep in data such as Uber, Twitter and Tesla, the use…

Note: This article was written and originally published in October 2016. I have reviewed and refreshed the content to reflect changes with the tools used in this article in 2020. The source code and data used and mentioned in this article are available from https://github.com/wyswilson/jobsearch-engine

Good search is about ensuring that users find what they need. It can be something as simple as the contact details of a restaurant that someone needs to make a booking to something more life-defining such as exploring for new jobs. …

In the first and second part of this 3-part article, we covered the importance of knowledge about the different types of data and the roles that they play in data solutions. Equally as important is ensuring that your business invests in the people who have such knowledge and know what to do with that knowledge in building the right data infrastructure and assets for data professionals to work with. We also discussed the importance of ensuring that your products are ready with the right data and that your budget permits you to hire the right people to work on the…

This is the second part of a 3-part article on the four hurdles to creating value from data. In the first part, we gave an overview of the four hurdles. We posited that the main challenges in extracting value from investments into all things data boil down to who (and to a lesser extent what) the businesses invest in and the order in which they happen. In this second part, we will discuss in detail the first two hurdles (of the four in total) that organisations need to overcome in order to get data working for them:

  • Lack of appreciation…

Data provide organisations the new platform in the 21st century and beyond for innovation of products and processes — there is no arguing that. From identifying bottlenecks in departmental processes and reasons behind marketing campaigns under performing, to using data to inform and improve products and services in ways that were not previously possible, everyone knows data is important. However, that is where the good vibes end. Knowing that the future of your organisations rest on data versus being fully aware of what it entails are two very different stories.

Photo by Drew Beamer on Unsplash

When we consider reports such as this, this and many…

A previous post on retrieval, precision and recall discussed the distinction between the three important concepts in search. In another post, we described the two main approaches for retrieval, and saw that, for faceted search, retrieval with high precision is the foundation for a good search experience. In other words, if we look at the results in its entirety as a set and not be distracted by the ranking, how relevant is each and every single result. A highly precise retrieval contains very little to no irrelevant results. …

We have spoken about the fact that relevance is subjective. However, this does not mean that lines need not be drawn. In the area of search, there are generally two approaches of bringing results back given a user query.

The first one, which partitions the entire collection into relevant versus irrelevant documents, is known as set retrieval. If the system subsequently decides to order the relevant documents that it returns to you, it should be for the purpose of distinguishing the more relevant results from the less relevant ones.

“relevance ranking should help distinguish more relevant results from less relevant…

We make decisions on a daily basis, from something as innocent as figuring out what to have for lunch to deciding on whether to go for that last bid or not in a property auction. In business settings, sub-optimal decision making can lead to bad experiences for users or customers and financial losses.

Photo by James Lee on Unsplash

It is worth emphasising at this stage that the core of the matter is not about avoiding mistakes at all cost or criticising incidents after the fact. Rather, sound decision making is about preventing the kind of undesirable outcomes that would have been picked up and avoided…

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store