Google is not the only thing !

While Population Explosion was one of the evils in the last couple of decades, Information Explosion is the emerging (or perhaps emerged) evil during this decade and the coming decades. There is ocean of data available on the web and finding the right information given a search query is an enormous and a herculean task. To solve the enigma of search companies like Google, Yahoo, Microsoft have search engines which are continuously tuned with new information coming in round the clock. It is a known fact that the size of the world wide web is increasing rapidly and Google is the father of all search engines that gets you the right information you are looking for !

Remember Osmosis process, where the molecules pass through a semi permeable membrane from a lower concentration to higher concentration. It is this process that I underwent over a decade ago by moving from a Tier 2 city to to the silicon valley of India i.e Bangalore. Definitely, to pass through the semi permeable membrane was the hardest thing. After landing in India's Silicon valley, I learnt that, Google was an answering machine.  Any doubt or Any question in mind? Rather than asking any one, Just ask Google ! I wondered at times that Google should have been named as Guru. Google has taken the place as walking encyclopedia and it has found its place in Oxford dictionary as well.  

After using the Guru aka Google for over a decade, it has now become a Genie - One who fulfills your wish. I have observed for quite some time that whenever I type the first word or second word in Google, there comes the auto suggested words that are more or less the ones that I was looking to search. And, this has happened many a times. Another incidence, that has happened, I looked for Mahindra Scorpio the soon to be launched face lift version. After some time, I saw an ad that suggested about winning a Scorpio by enrolling in to a contest. I had a wishful thinking to buy a Scorpio, here the web suggested me an option to get it. Google might have started reading your mind or rather subconscious mind ;)

A decade of habit of searching for information on Google has led to a strong belief that Google has the answer for everything. Searching has moved away from giving relevant information to predicting what information you are looking for. And, that's what Google is getting better and better day by day. 

Information is available in various forms, the same information might also be scattered on different platforms. We are now in an era of Information Explosion. A midst the plethora of information, searching or predicting what information you need is like searching needle or rather tiny needle in a haystack. And, it is this tiny needle that may get lost during search and even Google may miss it.

With the strong belief that Google can answer my requirement - Noise Cancellation Headphones with mainly the mic that does not transmit the noise to other end and also within a budget of ₹. 1000 . I searched on Google, I did not find the intended result.  I changed my query and tried again (Google) ,  I did try to find on Amazon, Snapdeal but I did not get what I intended in the top results. I gave up my  online search and inquired my friends and also posted on Facebook. None of them gave me a answer that I was looking for, though there were some options posted with the likes of Bose, Senheiser, JBL etc, but not within the budget.

A casual walk to the landmark book store during a weekend, got me what I wanted - Panasonic-RP-HM111PP-W. This product did not turn up online without the word 'Panasonic'  which I realized later. Also, the price that I paid was Rs. 599 and later I learnt the online price was Rs. 479, clearly a discount of over 20% was missed in this deal! Another such instance happened when I was about to buy a Vegetable Cutter  and the online price was around ₹1100 and a chance encounter at a Big Bazaar was ₹ 995.  Finally, it was a zero sum game between the two deals, however the bargaining opportunity was clearly missed by the father of searching engines or any other shopping sites in both the cases.( I do not know if SEO would have helped Panasonic to get me what I wanted). It is evident that the search engines face a gargantuan challenge to get the right product out of millions of product available using a query of few words.

Internet has evolved - world wide web, search engines, online forums, social networks, shopping sites etc.  To get the right information you need to know the right search query and the right platform to search for. Search might be thought to be a solved problem but the huge addition of information everyday makes it unsolved for every query that is searched for. Perhaps, Pranav Mistry's Sixth Sense Project would have helped  to take the search to the next level. And, People would have banned cell phones while entering the shopping zones;). But, Mistry has moved away from the Sixth Sense Project . 

For every query there is definitely Google, but sometimes a walk down to your nearby store makes you realize Google is not the only thing !

In an Uncertain but Predictable World

Uncertainty! In a world surrounded by corruption, scams, recession, job loss etc., the world we live in has become more volatile and highly uncertain. Amidst the uncertainty we are always curious and intrigued to know what happens next, tomorrow and so on and so forth. The mind is always speculative, take the example of news these days – Intense Speculation! We are very much eager to know what would happen to life – that’s why Astrology! We are very much eager to predict the stocks – that’s why Stock Trading. We are very much eager to  know the outcome of a match – that’s why Betting (And, Fixing)!Our mind continuously debates to predict the outcome of something or the other. 

Prediction has been existing since the Roman Republic where the tracking of all adult male fit for military service were maintained in records and since then existed the census. The word ‘Census’ is derived from the Latin word ‘censere’ which means ‘to estimate’. In the year 1880 the census of the American population took over 8 years, the next census was estimated to take over 14 years. This gave impetus to sampling techniques. Sampling techniques are used in various places like to predict the outcome of an election – the field known as Psephology, to forecast the weather(predicting seasonal changes goes back to as old as 25000 years ago), to determine the behaviour of consumers by taking a sample survey and so on and so forth. How to decide a sample is itself a challenge and could give incorrect outcomes chosen a wrong sample. Various sampling techniques have evolved over a period of time. These have been applied to various available data forms. But with the advancement of technology, digitization of data, Plethora of challenges and opportunities exist to apply the prediction techniques to the emerging field known as Big Data. We have unleashed and solved myriad of problems that could have been never solved without technology and digitized data.

The emergence of new technologies viz. cheaper machines, hard disks, Hadoop, No SQL data bases, Artificial Intelligence - Artificial Intelligence – Machine Learning, Natural Language Processing and Linguistics have all aided to solve the problem of Big Data mainly to predict the outcome of a particular problem. Let us look at some of them to understand the impact of these on the life of people.

Recommendation:  With the advent of social networks, the recommendation engines are seen to be handy tool for marketers to promote the products. Take for example of 

Facebook: There are various recommended things viz. ‘You may know’ (person), ‘Suggested Group’, ‘Add to news feed’, ‘you may also like’, etc. All this is possible because by profiling the person to whom these are recommended. If you have page, it also give you an analytics about the demographics of the user and helps you identify your target customer profile. It also suggests the place that you currently live in based on other friends whom you are connected with. Because of recommendations, I am able to connect to friends whom I might have missed due to the distance gap.

Linkedin: ‘People you may know’, ‘recommended news’, ‘jobs you may be interested in’, ‘Groups you may like’, ‘Companies you may want to follow’ and so on and so forth. This helps you know where jobs are available, which are your relevant companies depending on your profile in real time. The idea of searching jobs or companies is kind of vanishing.

IMDB: International Movie Database – a recommendation engine for movies. It is built using an algorithm that rates the movies based on the votes of the users. It becomes easier to know what movies to watch and what can be left out!

Likewise, these recommendations are used on most e-commerce sites like amazon.com, flip kart and many other places to recommend you a particular product based on your usage history.

Markets

Stock Market Predictions - There have been models built using twitter to predict the stock market. Some of the references are ‘Twitter Mood Predicts the Stock Market’ , ‘Tweet Predictions’ . In fact, I got an opportunity to work on similar problem in my previous firm known as ‘Real Time Intelligence’. The idea was to predict the behaviour of the company so that the fund managers can decide on which company the investment should be made. Using the same problem, it was extended to health care, Enterprise Risk analysis and many other domains. We used new that was aggregated form disparate sources to predict the trend and also tweeter was used as well. Even there have been studies using Google Trends to predict the markets. References are here , here and here.

Algorithmic Trading – If tweeter and news can help to predict the outcome of the stock value of the company. Then algorithmic trading helps you invest in large number of companies based on the algorithms  you choose. Of course, there are guidelines and in some instances these algorithmic trading has halted the stock exchanges due to a bug in the system too. But, this helps to invest on large scale which was humanely impossible to invest.

Human Resource

Predicting Employee Exit – Several companies are using the tools to predict whether an employee is about to exit and these tools help to engage the companies to retain the talent. Training the employee is quite expensive for organization and retaining the employees is one of the agendas for the HR.

Predict Performance before Employee Joins – There are also tools that predict whether the employee is going to be successful in his job before he or she actually joins the company. Now, this might be incredible. Thanks to the data trails left by the employees on the web.

BMI Prediction – Usually, to calculate BMI you take the weight and height of the person and then calculate the BMI. How about looking at the face in a photograph and then calculating the BMI? Not possible then click here to believe it. 

Farecast  (Acquired by Bing) is used to predict when the best time to purchase a flight ticket is, Decide is used to predict when to buy electronic products and likewise the list of prediction applications is endless. One of the interesting predictions that I came across was about ‘Algorithms Calculate a Couple's Chances of Having a Baby via IVF’. And, the best ever has been by IBM Watson’s DeepQA project.  No industry is left without leveraging the Big Data technologies and Prediction algorithms. This has opened the flood gates of opportunities to the so called Data Scientist, Entrepreneurs and Academicians to solve challenging problems and to make an Uncertain World in to a Predictable One!