In an Uncertain but Predictable World

Uncertainty! In a world surrounded by corruption, scams, recession, job loss etc., the world we live in has become more volatile and highly uncertain. Amidst the uncertainty we are always curious and intrigued to know what happens next, tomorrow and so on and so forth. The mind is always speculative, take the example of news these days – Intense Speculation! We are very much eager to know what would happen to life – that’s why Astrology! We are very much eager to predict the stocks – that’s why Stock Trading. We are very much eager to  know the outcome of a match – that’s why Betting (And, Fixing)!Our mind continuously debates to predict the outcome of something or the other. 

Prediction has been existing since the Roman Republic where the tracking of all adult male fit for military service were maintained in records and since then existed the census. The word ‘Census’ is derived from the Latin word ‘censere’ which means ‘to estimate’. In the year 1880 the census of the American population took over 8 years, the next census was estimated to take over 14 years. This gave impetus to sampling techniques. Sampling techniques are used in various places like to predict the outcome of an election – the field known as Psephology, to forecast the weather(predicting seasonal changes goes back to as old as 25000 years ago), to determine the behaviour of consumers by taking a sample survey and so on and so forth. How to decide a sample is itself a challenge and could give incorrect outcomes chosen a wrong sample. Various sampling techniques have evolved over a period of time. These have been applied to various available data forms. But with the advancement of technology, digitization of data, Plethora of challenges and opportunities exist to apply the prediction techniques to the emerging field known as Big Data. We have unleashed and solved myriad of problems that could have been never solved without technology and digitized data.

The emergence of new technologies viz. cheaper machines, hard disks, Hadoop, No SQL data bases, Artificial Intelligence - Artificial Intelligence – Machine Learning, Natural Language Processing and Linguistics have all aided to solve the problem of Big Data mainly to predict the outcome of a particular problem. Let us look at some of them to understand the impact of these on the life of people.

Recommendation:  With the advent of social networks, the recommendation engines are seen to be handy tool for marketers to promote the products. Take for example of 

Facebook: There are various recommended things viz. ‘You may know’ (person), ‘Suggested Group’, ‘Add to news feed’, ‘you may also like’, etc. All this is possible because by profiling the person to whom these are recommended. If you have page, it also give you an analytics about the demographics of the user and helps you identify your target customer profile. It also suggests the place that you currently live in based on other friends whom you are connected with. Because of recommendations, I am able to connect to friends whom I might have missed due to the distance gap.

Linkedin: ‘People you may know’, ‘recommended news’, ‘jobs you may be interested in’, ‘Groups you may like’, ‘Companies you may want to follow’ and so on and so forth. This helps you know where jobs are available, which are your relevant companies depending on your profile in real time. The idea of searching jobs or companies is kind of vanishing.

IMDB: International Movie Database – a recommendation engine for movies. It is built using an algorithm that rates the movies based on the votes of the users. It becomes easier to know what movies to watch and what can be left out!

Likewise, these recommendations are used on most e-commerce sites like amazon.com, flip kart and many other places to recommend you a particular product based on your usage history.

Markets

Stock Market Predictions - There have been models built using twitter to predict the stock market. Some of the references are ‘Twitter Mood Predicts the Stock Market’ , ‘Tweet Predictions’ . In fact, I got an opportunity to work on similar problem in my previous firm known as ‘Real Time Intelligence’. The idea was to predict the behaviour of the company so that the fund managers can decide on which company the investment should be made. Using the same problem, it was extended to health care, Enterprise Risk analysis and many other domains. We used new that was aggregated form disparate sources to predict the trend and also tweeter was used as well. Even there have been studies using Google Trends to predict the markets. References are here , here and here.

Algorithmic Trading – If tweeter and news can help to predict the outcome of the stock value of the company. Then algorithmic trading helps you invest in large number of companies based on the algorithms  you choose. Of course, there are guidelines and in some instances these algorithmic trading has halted the stock exchanges due to a bug in the system too. But, this helps to invest on large scale which was humanely impossible to invest.

Human Resource

Predicting Employee Exit – Several companies are using the tools to predict whether an employee is about to exit and these tools help to engage the companies to retain the talent. Training the employee is quite expensive for organization and retaining the employees is one of the agendas for the HR.

Predict Performance before Employee Joins – There are also tools that predict whether the employee is going to be successful in his job before he or she actually joins the company. Now, this might be incredible. Thanks to the data trails left by the employees on the web.

BMI Prediction – Usually, to calculate BMI you take the weight and height of the person and then calculate the BMI. How about looking at the face in a photograph and then calculating the BMI? Not possible then click here to believe it. 

Farecast  (Acquired by Bing) is used to predict when the best time to purchase a flight ticket is, Decide is used to predict when to buy electronic products and likewise the list of prediction applications is endless. One of the interesting predictions that I came across was about ‘Algorithms Calculate a Couple's Chances of Having a Baby via IVF’. And, the best ever has been by IBM Watson’s DeepQA project.  No industry is left without leveraging the Big Data technologies and Prediction algorithms. This has opened the flood gates of opportunities to the so called Data Scientist, Entrepreneurs and Academicians to solve challenging problems and to make an Uncertain World in to a Predictable One!
 

If You Cannot Convince, then Distract!

50,000!!! That’s the number of thoughts per day that a human processes in its brain. Perhaps some may be repetitive and the average is 0.5 thoughts per second. Incredible!  This tiny brain that weighs around 3 pounds and has 75% water boasts of processing 50000 thoughts per day. I am not going to dissect the anatomy and the science behind the brain functioning here.  But, delve in to how information has overloaded the brain and distracts the mind. 

Roti, Kapada, Makaan (Food, Clothing , Shelter) has been the basic necessities of life. Adding to these necessities, there are many other things like TV, Web, apps, social networks, gadgets, mobile phones and many other things.  It is this addition of things to life that has made life sometimes feel miserable and sometimes feel connected.  Take the example of a news channel on TV, the news screen has at least 5 varieties of information and goes up to 10. The information includes Time, Channel Name, Breaking News, Just in News, Top Story, tickers that scrolls the news, important news schedule and stock prices and so on and so forth. The main news that the anchor is presenting is surrounded by these many information and your eyes and mind is always looking for additional information on the screen without your knowledge.

Let us look at the information on a social network site.  When you login to your page, you have recommended friends, the wall, trending topic, the news feed, sponsored ads, the groups, the pages etc. From an era of searching the information we have moved to an era of scrolling for information on these social networking sites. Amidst the scrolling, there is humongous amount of information that distracts you and deflects your purpose of visiting the site.

Let’s move to email. The relevant mail is not to be seen at all. You need to dig out the relevant mail amidst the marketing mails all around. New features are accommodated in mail boxes like social, updates, automated unsubscribe etc.  A recent article by muHive mentions the use of emails for marketing purpose is less useful.  But still people thrive on mass marketing whether it’s useful to the person or not. I just hate to see more ads or marketing mails in my mail box then real mails. 

Duplicity of information is another nuisance. The same information is posted on every other social network. Visit any site or blog there is an option, share on Facebook, twitter, pinterest, googleplus, and some pages have buttons from almost all social networks on the earth where you can share. And, it is quite annoying to read the same information since everyone is on every other social network. People share information that I have shared X information on Y network so please check. Irritating? The mantra is simple, push the information anywhere and everywhere because it’s easy and cost is almost zero!  It was interesting to see the purpose of social networks to connect people. Now, it has moved on to make people consume all sorts of information whether they like it or not. Another nuisance from these social networks is, automatically your profile gets created without even your knowledge! This is insane and furious!

The number of online accounts with separate credentials easily goes over 50 accounts for an individual. Managing this information is itself quite challenging. With the enormous amount of data that is generated on different mediums, the eyeballs that  screens through information easily surpasses over the number of thoughts per second which in turn means the brain has to process at a speed faster than the speed of thought. This is quite impossible and leads to deviate from your original thoughts.

Even Arjuna, hit the fish eye with his bow and arrow by looking at the reflection in water, would find it hard to navigate through the information. Earlier, we used to search for information, now information is being delivered to you whether you like it or not. The marketing personnel are going with the strategy of pushing the information with the motto, 'if you cannot convince, then distract' . For the user Call it information Overload, Redundant Information, people will be ignorant about Information and we are entering in to Information Ignorant World which can also be called as Attention Deficit Disorder :)

Communication Conundrum

It was a bright sunny day, the sky was clear, the crowd was very enthusiastic and it was a full packed house. This is not about the cricket match, but it was a lecture hall meant for a Guest lecture by one of the leading HR Professional from an MNC.  The talk was meant for graduating students focusing on their soft skills, preparation for interviews, personality development etc. Little did we realize the importance of these skills then, but I do realize that these are most sought out skills in the industry. Among these, one of the most important topics that covered was communication. 

Imagine a communication with a kid, it starts like 'ali-lee-lee, ali-lee-lee' :). As the kid grows, communicating to the child changes and as the kid becomes an adult, parents ought to treat them like a friend and communicate in a different way. Communication is contextual; it is quite different in a business set up, professional set up, friends, family and depends on each individual based on their behaviour. Communication is a part and parcel of a daily life and 80% of the time it is the communication that leads to effective business, ideas or requirements that helps to design a system. 

Communication has transformed enormously in the recent decades. The change has been so drastic that it has created communication gaps like generation gaps. The older generation is unable to adept to newer form of communications that technology has created.

In the olden days, if someone was about to invade the kingdom, fire used to be the form of communication indicating the threat. Pigeons were trained to pass on the information. These forms of communication are history, thanks to the invention of telegraph and telephone. 

'XYZ serious, start immediately'. ('XYZ might be dead as well that wouldn't have been communicated') This is not an SMS but this is how a telegram used to be sent and delivered by the postal department. And the concerned kin used to travel to the destined place. Telegrams were used in the military for sending coded information. Though most of us are not using telegram, it exits. (IndianExpress). It appears that it will become extinct soon. (LiveMint-1 LiveMint-2)

Later, telephone revolutionized communication and reduced the distance among people. If computer was the wheel, then web is the vehicle that carries the information instantaneously. With the advent of the social networks, you know who has checked in where, what is their location, which restaurant they are in. You also know if someone has run away and got married :). You can even find kumbh mele mein bichade hue bhai etc. etc. Without the use of telephone, web and social networks business seems to be impossible. No doubt these technologies have almost killed the post, fax and telegram.

No one can deny that technology has advanced and has made the information accessible instantaneously. It has also created a vast number of mediums or channels to communicate. Take for example,

• Product or Service: If I need to report a complaint about a product or service, the mediums that are available are email, online, customer care, social networks like twitter, Facebook where companies like flipkart are leveraging to its best. But, for example, recently I had an issue with a nationalized bank about an ATM card and I lodged a complained through email. Apparently, the issue was closed without even my knowledge but the issue still existed. I even tried toll free number; they redirected me to the branch. After commuting to the branch thrice it got sorted out.

• Office: At office, the professional means of communication are email, Skype, chat, phone or be present physically to present the ideas.  Under different scenarios each of the tools needs be used effectively. If by chance you use phone over chat or email that may not be appreciated. Knowing when to use what is an art in itself. Also, you need to be worried about whom all needs to be addressed in an email and who all needs to be out. Also, whom to put in 'CC 'and whom to put in 'To'! These are never taught in any school and its learnt only hard way!

• Individuals: If you are meeting a doctor, he or she may prefer only face to face communication and any communication over phone or email is outright rejected. Each individual may prefer their own way of communicating.

• Rumours: If there is any news that is prominent, then suddenly there is a reaction over the social networks and sometimes it leads to rumours’. I remember the 26/11, it was said that terrorists are absconding in to the city.  There are cases of false deaths about famous celebrities and they come alive to prove their mere existence. It just creates confusion and you really do not know how to react!

No doubt communication medium has been a spoiler but it’s playing a double edged sword, for example

• Traffic Control: With the enormous increase in traffic, the earlier mode of communicating to traffic by the traffic police by hand signals was given a significant boost with the advent of automated traffic signals. With the ever increase in traffic, things would have been disastrous without the traffic signal.

• Banking:  The scenario mentioned earlier might get resolved. Internet banking has made obsolete the visit to a branch. You can transfer amount to any one online.

• Products Purchase: Almost every company is on the web selling their product and has made your life easy by giving you the option of purchasing the item online.

• Education : There are courses available online from renowned universities and one can educate oneself by registering online and taking up the course for example : Coursera

Likewise there are many innumerable instances where web has played a prominent role to make life easier but the underlying communication conundrum is increasing due to increase in the number of communicating channels. The same information might be present in different channels and the data is exploding exponentially, and the phenomenon called 'Big Data'. 

Businesses are thriving on these communication channels and the world has become small. Due to these channels, people across countries are connected and the barrier of language is no longer a challenge.

People are working on solving the challenges pertaining to Big Data and have evolved with several solutions. From an individual perspective, I wonder if there is a need to consolidate the number of communication mediums and use for example only virtual medium. But, I don't think if consolidation of communication medium would ever take place and there shall be only increase in the number of channels.

The conundrum shall always remain as the saying goes if you cannot convince, confuse!