Assume you’re in the real estate business or own agricultural land and want to discover every detail about a bit of land/location sitting a thousand miles away. Doesn’t that sound a little too intricate? Well, Telescope helps you to do it in a snap! 

What is Telescope?

It’s a next-generation AI satellite image segmentation solution capable of resolving complex business and significant operational requirements. Telescope uses a machine learning framework to classify information in a digital image (i.e., buildings, roads, grasslands). It then generates output segmentation data, which can be utilized for diverse business purposes such as pattern identification and object tracking.

Telescope emphasizes the concept of leveraging AI and has established a software package that utilizes cloud services to allow you to extract valuable data pertaining to your business. This platform lets users perform real-time image analysis on high-resolution satellite images and view adjacent locations with the accurate coverage percentage of greenery, land, buildings, and water bodies.

• Telescope uses a cutting-edge combination of Computer Vision and GIS technologies that allows to automatically retrieve high-resolution satellite images of sites with up to 100 square km of magnification
• It enables single or multiple pairs of lat-long parameters or location names in various formats
• Uses a Deep Learning Feature Pyramid Network (FPN) model with a point rend module for precisely predicting the label maps
• It will allow you to assess variations in water bodies or land shapes like dams, rivers, deserts, and mountains

Telescope provides a broad range of real-world applications such as monitoring deforestation, urbanization, traffic recognition of natural resources, urban planning, etc. This novel technology can also be used in critical missions for the uniformed forces and in monitoring catastrophic catastrophes like volcanic eruptions, wildfires, and floods, among other things. With an intuitive interface and a substantial reduction in manual effort, it automates land surveys and minimizes the requirement for data collection associated with mapping and construction operations.

In other words, this solution enables faster and more accurate data processing, reducing time and cost, and is very handy for professionals. Furthermore, the solution’s atomized approach yields quicker and more accurate outcomes, with much less reliance on third-party data sources.

What is More Exciting About Telescope?

• Its segmentation technique automates the process of extracting structures like land, lake, etc., from satellite images without the requirement of any advanced skills in Computer Vision or geospatial data
• It has been built using advanced image processing algorithms, offering the most accurate results and seamless integration with real-time API services
• The solution can be readily incorporated into current corporate GIS systems or used as a standalone solution to handle geographic data
• Exports multiple high-resolution images leveraging Google Maps as a default source for satellite images
• Highly interactive dashboard for reporting and visualization
• Highly accurate percentage-wise representation of the areas covered in various categories
• Save your planning time up to 70% and costs up to 30%

A Bigger Picture Beyond the Flaws

Whether it’s a mission for any uniformed forces, monitoring catastrophic events, feasibility analysis for mobile tower setup, or planning a smart city, Telescope is a one-in-all solution for your business need. Request a demo to learn how to leverage this solution for your business need. Be ahead of the technology crux! 

Product Pricing

What elements do we consider when selecting a product’s price? Is it entirely dependent on customer demand? Firms & marketers are constantly striving to unfold the relationship between sales demand and price fluctuation to find a pricing point that is best for their products. Let us look at price elasticity to reveal the facts behind this relationship.

What is Price Elasticity?

Price elasticity of demand (PED) is an economic measure representing the responsiveness, or elasticity, of the quantity demanded to the change in the price of a product or service. To simplify, it is the ratio of percentage change in quantity demanded of a product in response to the percent change in its price.

Most markets are sensitive to the price of a product or service, the cheaper the product higher the demand, and vice versa. While this need not be true for all products and services, price elasticity as a quantifiable phenomenon shows precisely how sensitive customer demand can be for a product price. For starters, let us look at questions professionals in marketing try to answer when determining the elasticity of their products:

• How much more can you sell by lowing the product price?
• How will the rise in the price of one product affect sales of the other products?
• How will a decrease in the market price of a product affect the volume of production & market supply?

What Is Price Elasticity of Demand & Supply?

Let’s look at Coca-Cola to try and understand these concepts. Assuming that a bottle of Coca-Cola regularly costs $1, if the price surges to $2, it will likely result in a dip in demand as most people would consider it expensive. On the other hand, if you drop its price to 10¢, you will notice a significant rise in its demand.

Price elasticity springs from the fundamental economic law of supply and demand:

• Cheaper the product, the higher the demand
• And the more expensive a product becomes, the lower the demand

How likely is Sales Demand to Change When Price Changes?

Price elasticities are usually negative; when our price decreases, our sales demand increases. It makes sense, doesn’t it? However, positive price elasticities are found in rare cases where products do not conform to the law of demand. 

Yes, these case scenarios happen when we have Veblen goods. Veblen goods are typically high-quality, exclusive, and a status symbol. Example – Louis Vuitton

What will the Price vs Consumer Demand Curve Look Like?

The chart above shows that a 33% increase in price point decreases consumer demand by a million, whereas demand doubles if we drop it by 50%.

Mathematical formula to calculate the price elasticity –

Price elasticity (E) is the percentage change of an economic outcome (which is generally the number of units sold) in response to a 1% change in its price:

Where:

% Change in Quantity Demanded = (New Quantity – Old Quantity)/Average Quantity,

% Change in Price (P) = (New Price – Old Price)/Average Price

For example, let’s say that a clothing company raised the price of a coat from $100 to $120, and the 20% increase in price caused a 10 % decrease in the quantity sold from 1,000 coats to 900 coats. Formulating these numbers gives you a price elasticity of demand which is 0.5, as mentioned below:

-.10 / -20 = -0.5 or 0.5

Types of Price Elasticity

1. Perfectly elastic: Minimal change in price results in a substantial change in the quantity demanded
2. Relatively elastic: Minor changes in price cause a tremendous change in quantity demanded (E > 1)
3. Unit elastic: Any change in price is matched by an equal change in quantity (E =1)
4. Relatively inelastic: Substantial change in price causes minor changes in demand (E < 1). Gasoline is a good example. As an essential commodity, demand stays relatively the same even with an increase in its price.
5. Perfectly inelastic: Quantity demanded does not change even with a price change. There is no elasticity of demand or supply for these products. Perfect inelasticity happens with products or services where the consumers do not have any other substitute goods. For example, food, medication, etc.

Factors of Price Elasticity:

1. Purchase probability: The likelihood of a customer purchasing a product from a particular category. For instance, in the case of beer, where there may be various brands with differing prices from the same product category, purchase probability determines how likely a customer will buy one brand instead of the other. Assuming we can compute the aggregate price of a category and according to the rule of demand, the higher the price, the lesser the demand, and the likelihood of purchasing beer decreases with the aggregate rise in its price. Calculating the price elasticity reveals this change in demand.
2. Brand choice probability: Brand choice probability defines the customer’s choice. If you work for Oreo, you are more likely to be concerned with Oreo’s brand compared to the overall biscuit sales. That is why marketers focus on persuading clients to select their brand over their competitors. When the cost of a product from a brand increase, the chance of purchasing that brand decreases.
3. Purchase quantity: Purchase quantity represents a customer’s expected purchase.

Analytical Model Implementation

We understand the business context of price elasticity and how important it is for every business to maintain a healthy financial sheet. The following operational question is how to solve / implement it. We have used Linear regression, either the Ordinary Least Square (OLS) or Recursive Least Squares (RLS) method, to predict quantity from the price change over time for any specific product.

Linear Regression Equation:

Where,

β is the beta coefficient (slope). Generally, beta is negative with respect to quantity sold, except for few exceptions where it can be positive.

Multivariate Linear Regression:

If additional supporting factors are directly linked with sales, we may utilize them in multivariate linear regression alongside the price variable. 

Price Elasticity Score:

To get the elasticity value of a product, we need to use equation #1 from above:

As price & unit have a linear relationship, beta (dy/dx) will remain constant. Hence, we can use the average value of price & units to get the elasticity score.

Business Application:

Once ready with the price elasticity model, the next step is to apply the business learning to reveal a product’s sensitivity in the market

The above chart shows that the Samsung-65 Class LED TV with a negative price elasticity of -17.68 will have 170.6% more demand with a 10% drop in its price or lose 170.6% of its sales demand with a 10% rise in price.

Whereas the Sony XBR-X850E-Series 75-Class TV with a positive price elasticity of 7.19 will notice a 71.2% drop in its sales demand with a 10% price reduction and a 71.2% boost in sales demand with a 10% increase in its price.

What Next?

New problems arise the moment you solve a business problem. For instance, what is next? Or is there a better approach? What can we do additionally to improve the implemented solution? The case is the same for price elasticity models. We see that changing the price of a product affects its sales. But what happens with a change in the price of a competitor’s products? The phenomenon that causes a shift in demand for one product from a change in the price of competing products is known as Cross-Price Elasticity. But more on that in the next blog!

References

• https://towardsdatascience.com/calculating-price-elasticity-of-demand-statistical-modeling-with-python-6adb2fa7824d
• https://github.com/susanli2016/Machine-Learning-with-Python/blob/master/Price%20Elasticity%20of%20Demand.ipynb
• https://365datascience.com/trending/price-elasticity/
• https://hbr.org/2015/08/a-refresher-on-price-elasticity
• https://corporatefinanceinstitute.com/resources/knowledge/economics/price-elasticity/
• https://ileanacabada.medium.com/price-elasticity-of-demand-using-linear-regression-in-python-part-1-a28c844c1656
• https://medium.com/geekculture/price-elasticity-of-demand-using-linear-regression-in-python-part-2-8adb654328e7

After almost two years of state-imposed lockdowns, it is nice to see children back in school again. Roads and trains are congested again. Markets and movie theatres, restaurants, and pubs report a surge in footfalls. Evenings outside the four walls of a home are back again, with masks, of course – they’re the new dress code.

So does that mean it’s only a matter of time before offices open and we go about business as usual? Not entirely, a recent survey from LiveCareer claims that 81% of employees today enjoy working remotely. While most people might think that the future of work will host hologram meetings or have robots who cook fancy office meals, the future workspace boils down to the basics – empowering employees with the freedom & flexibility to work from anywhere convenient.

Vaccines are a New Lease on Life

The global medical and health fraternity and frontline workers operated through the pandemic to fight the invisible enemy. Scientists worked round the clock to identify the deadly virus, conducted methodical clinical trials, achieved regulatory approvals from nodal bodies, and manufactured vaccines in record time.

Today, more than 11.4 billion doses have been administered in 184 countries. And this global drive has been successful in controlling the spread of the virus and the severe illness by building immunity. The positive results of the worldwide vaccine drive were visible during the Omicron surge. The vaccine with a booster dose reduced the chance of hospitalization and death by more than 90% and made the virus less fatal.

After 6,210,719 deaths, according to the WHO dashboard as of 22^nd April 2022, and two challenging years later, the vaccine drive gives us hope that ‘normal’ is not lost forever. But can we say the same about our work life?

Remote work is not a new trend. What was a slow drift for p.c. users who worked off-site & after hours simply accelerated with the pandemic, forcing remote work as a large-scale trend that is here to stay. But how did the perception of workspace change over the last two years, and where is this trend going?

Benefits of Working from Home/Anywhere

There is no denying that the boundaryless work culture is gaining momentum. In contrast to spending time, money and energy being stuck in traffic with reckless drivers, the popular choice leans in the favor of serene waterfront open-rooftops or working from beach cafés that offer beautiful sunsets. Below are some other benefits that entice employees to make the most of work beyond boundaries:

Firms are replacing vertical hierarchies with horizontal networks due to rapidly changing market conditions and global competition. Thanks to connectivity, brought to you by the advancements in modern technology, more and more employees report that working from a place of their comfort has positively affected their work-life balance with a higher sense of security. In addition to acquiring new career-related skills, people now better understand expectations, inducing clarity & collaboration.

“Tier 2 & Tier 3 cities are benefitting with an increase in the talent pool and decreasing the dependency on Metro & Tier 1 cities.”

Working from offices was once the norm in the workplace. As a result, the talents of Metro and Tier 1 cities benefited more than the talents in Tier 2 and Tier 3 cities. The pandemic has resulted in a cultural shift in the workplace. Thanks to technology advancements, talent can now be identified and hired from anywhere. This is why most companies across India are now putting a greater emphasis on talent and hybrid working practices.

Work-from-home, work-from-anywhere, and remote work are widely adopted hybrid working styles that have enhanced productivity while allowing skilled employees to work effectively with a greater comfort level. Talents no longer need to be away from their families for work in Metro or Tier 1 cities. They can set up their workplace in their hometown (in tier 2 and tier 3 cities) and work from anywhere.

The Future of Work in the New Normal

The pandemic has made revolutionary changes in the working style of the business world. Individuals are now better equipped to work and collaborate remotely. Digital technology has transcended age and generation at the workplace and is now everyone’s best friend.

Businesses are rethinking the changes and reorienting working models for the future of work. The culture of remote work and the hybrid work model will continue as uncertainty looms in the coming months.

When asked if employees would like to return to the office, most responded by saying that they want their employers to let them work in a remote capacity indefinitely, even after the pandemic is over. Nearly 30 % of the respondents go as far as to say that they would quit their jobs if they were not allowed to continue working remotely.

The remote working model suits knowledge workers as they have demonstrated that they can be trusted to deliver from anywhere. These emerging work models also serve the startup and SME employers as their office infrastructure costs decrease substantially.

However, several large enterprises and corporate houses have slight apprehensions about remote working as they feel that the company’s culture may be at stake if their entire workforce is remote. Considering the sentiments across the industry, hybrid models will be the way to go, at least in the coming months.

Summing Up

The pandemic disrupted business and labor markets globally in 2020. The short-term consequences were severe as millions lost jobs and a global workforce adjusted to working from home. Two years down the line, the virus finally shows signs of moving to endemicity in a few parts of the world.

The good news is that offices have opened, retail, malls and F&B outlets are functioning, and transportation services are making a comeback. Travel in the international sector continues to be restricted. At the same time, employees worldwide have realized that spending a few days in the office aids in better collaboration and boundaries between work and life.

Life, to a great extent, is back on track. But business will take more time to reach the normal state of affairs. If LinkedIn is anything to go by, the global workforce is celebrating a hybrid work model committed by the likes of Google. And as uncertainty lurks around the onset of the next pandemic wave, embracing a boundaryless work culture might be more practical. We may not be returning to the old normal soon, but the new normal certainly awaits us.

Building responsible technology has become a top priority for enterprises across industries as understanding and scrutiny of the challenges connected with AI have grown. Responsible AI evolves from a “best practice” to the “high-level principles” and guidelines required to drive system-level change, building trust around the business world. Regulators are taking notice, also advocating for regulatory frameworks surrounding AI that include enhanced data protection, governance, and reporting standards.

Can AIOps Help Your Business?

The current business environment across industries is changing rapidly, causing software and systems to grow more complicated. Cross-functional teams; are expected to generate solutions more quickly and efficiently in these circumstances. Site Reliability Engineering (SRE) and Network Operation Center (NOC) teams in the DevOps setup; are continuously flooded with a mountain of data. At the same time, these teams are increasingly under pressure to handle many dashboards, frequent alerts, and signals from multiple tools, making it harder to identify the root cause of problems/instances. AIOps as a solution; assists these teams in finding solutions to the problems more quickly and effectively, allowing them to focus more on creative work and the software development process.

Businesses are intrigued to start investing/buying an AIOps solution in order to transform their business. But factually, many business leaders and technologists fail to understand that success with AIOps lies within the right preparation and not just limited to the technology adoption.

Resilient. Agile. Ease. – “Shift Left” makes it hassle-free for SRE and NOC teams in DevOps setup

Enhancing the application development lifecycle is a potential use case for AIOps. Applications are constantly changing as a result of digital transformation, with new iterations frequently surfacing each day. A domain-agnostic solution can be “Shift Left” in the lifecycle to provide observability and control to Site Reliability Engineering (SRE) and Network Operations Center (NOC) teams in DevOps set up that were previously available to ITOM and ITSM teams.

Why are businesses failing to take full advantage of AIOps capabilities?

Before discussing the bottlenecks of businesses implementing AIOps, let’s understand AIOps in a nutshell…! 

What is AIOps?

– According to Gartner, “AIOps combines Big Data and Machine Learning to automate IT operations processes, including event correlation, anomaly detection, and causality determination.”

It defines artificial intelligence in IT operations and workflow. It refers to the strategic application of AI, Machine Learning (ML), and Machine Reasoning (MR) technologies across IT operations to simplify, streamline, and optimize the use of IT resources.

In a nutshell – AIOps is designed to address the growing challenges of complex IT infrastructures and help organizations make the transition to a better, more efficient future. AIOps can be used to improve IT productivity and reduce costs, automate repetitive tasks, create a more predictive environment by collecting data from multiple sources and analyzing it using ML/MR technologies, reduce time-to-market for new innovations, etc.

Supporting this fact are the other key findings that say –

• 53% of AI adopters mentioned “lack of transparency” as one of their significant bottlenecks
• 54% of respondents expressed concerns about making bad decisions based on AI recommendations
• 55% of respondents worried about the liability for decisions and actions taken by AI systems

And the reason they fail to implement or take full advantage of AIOps!

Challenges: Top 5 Reasons why AIOps implantation Could Fail 

1. Incompatibility with existing tools: Is it feasible for your software systems to communicate data in an efficient way? In order to provide valuable insights, an AIOps-based analytics platform will need data from other software systems. The failure of your AIOps transformation could be due to interoperability with existing software platforms. 
   
   If current software systems prevent you from collaborating with other products or systems, it’s time to think about an IT transformation. Let’s assume you are using a digital service desk AI software to generate the tickets, and your current legacy tools fail to process the request. It will land you in trouble. Thus, make sure that the tickets generated by the services desk are forwarded to your legacy tools for analysis. If your existing devices are compatible with an AIOps based analytics platform, it will automatically process the events/instances from the service desk and generate actionable insights.
2. Lack of awareness to identify problem areas: You aren’t undergoing AIOps transformation simply to implement cutting-edge technology. The primary objective of employing AI in operations management services is to boost the productivity of your IT operations. Apart from keeping up with the current AI trends, it would help to concentrate on the areas where an AIOps transformation is required. 
   
   Some of your IT operations may be efficient enough that you don’t need the help of an AIOps-based analytics platform. Adopting AIOps can be costly; thus, it’s best to identify the major bottlenecks that reduce the ROI (Return on Investment). Even the best AIOps tools and products have specific use cases and cannot assist you with a problem right away.
3. Outdated strategies to train data: In order to achieve scalable results, you need to feed the AIOps-based analytics platform training data. Data serves as fuel for AI/ML algorithms, allowing them to learn about IT processes. Organizations fail to provide training data to AIOps-based analytics platforms, resulting in AIOps transformation failure. 
   
   Even matured businesses fail to provide AI/ML algorithms with sufficient training data to improve their performance. Your AIOps-based analytics platform will not deliver relevant insights if your training data is cluttered and contains many outliers. The organization data is constantly fragmented across many software platforms and is unstructured. AIOps-based analytics tools cannot perform to their full potential without a comprehensive view of the organization’s data.
4. Lack of performance metrics understanding: How would you know if something is wrong with your AIOps transformation? One option is to wait and see how the attempted AIOps transformation impacts your ROI. Another method for determining the value of AIOps adoption is to use performance metrics. If you discover inefficient AI DevOps platform management services in a timely manner, you may be able to do transition to an alternative transformation plan. The following are some of the KPIs that can help in measuring the impact of AIOps transformation:
   • MTTD (Mean Time to Detect): It refers to the amount of time spent investigating an IT incident. The MTTD should decrease if AI is used for application monitoring.
   • MTTR (Mean Time to Detect): It signifies the amount of time it takes to resolve an IT issue. The use of AI in operations management services should always result in a considerable reduction in MTTR.
   • Service availability: AIOps systems will constantly improve the availability and reliability of your services. If the availability of your services isn’t improving, it’s time to modify your AIOps strategy.
5. Reluctant or inability to adapt to the changing IT culture: You need to understand that AIOps adoption is not just limited to technological changes; it will dramatically change your IT culture. For instance, it would be difficult for your cross-functional teams to trust the decisions suggested by AI data analytics monitoring tools. So, understanding its capabilities and how it works is crucial for the teams involved in the process. You need to conduct the workshops and training sessions to educate them in a timely manner. In this regard, reskilling and upskilling the respective teams could give you faster results denoting cultural shift the way teams think and work for an organizational goal. On the other hand, you can leverage open-source AI/ML tools that can be tailored to match your current IT culture.

How to overcome challenges in AIOps implementation?

• Observe: Get started with real-time big data processing by identifying and collecting incidents/logs/alerts/request raised/event history from all underlying systems (application, network, infrastructure, etc.). 

• Think: Machine learning/deep learning techniques can be used to enable AI-based insights and recommendations. Start with AI-based insights, such as noise reduction through event de-duplication and grouping and real-time anomaly detection. Event correlation for causal analysis, automated RCA, application failure prediction, and change impact analysis are some of the more advanced use cases you can consider in the process.

• React: RPA, ITPA, scripts, and orchestrators can be used to initiate auto-heal/self-remediation operations. You need to have an experienced team or train them to manage the workflows using these methodologies.

• Learn: Allow AI-based learning to learn from previous incidents from both successful/failed attempts made in the process and use it to predict future outcomes to plan your next attempt. This approach is paramount to understanding your challenges in every possible way and learning how to overcome them strategically.

AIOps Market Report & Recommendations 

One of the critical factors driving market growth of AIOps is the growing demand for automation across industry verticals, such as banking, financial services, BFSI, healthcare, automation, IT, and logistics.

Businesses use AIOps to prevent and control security breaches by monitoring the activities and transactions of employees, customers, and external agencies. In accordance with this, the Covid-19 influence has resulted in widespread enterprise adoption of the work-from-home (WFH) trend, which is considerably contributing to market growth. When working remotely, AIOps is applied to improve information security and many other operational workflows effectively.

Various technological breakthroughs in cloud-computing solutions are also contributing to the AIOps growth. Businesses across industries are using cloud-based solutions for major business applications, performance, network, and security management that are gaining popularity at pace. AIOps also contributes to improving the overall efficiency of the infrastructure for service delivery with less manual efforts. In other words, considering the relentless improvements in the IT infrastructure, specifically in boosting the economies with extensive research and R&D initiatives, are expected to propel the market growth.

“There is no future of IT operations that does not include AIOps. This is due to the rapid growth in data volumes and pace of change (exemplified by rate of application delivery and event-driven business models) that cannot wait on humans to derive insights.”

~ Gartner

AIOps Market Size, Share and Trends – Industry Forecast 2022 to 2027

“The global AIOps market is expected to exhibit a CAGR of 21.2% during 2022-2027”

The past and current aftermath COVID-19 outbreak has given the AIOps market a significant boost. As AI-based IT operating solutions become more common, the industry has accelerated significantly, allowing businesses to reduce workflow and resource effort; allowing to focus more on innovation. During the pandemic, AIOps provided various benefits to businesses, including automating monotonous processes, delivering meaningful reporting, and improving risk management. As a result of these efforts, the industry’s post-COVID-19 repercussions have increased drastically.

 AIOps Market Size, Share and Trends

What are the AIOps trends in the U.S, Singapore & Germany? Where its market stands in these regions? 

Domain-agnostic tools is the next big thing in the U.S region 

In the United States, the domain-agnostic segment is expected to produce roughly USD 500 million in revenue by 2023. These solutions primarily rely on monitoring tools to collect data and respond to changing customer prerequisites. AIOps platforms are being used by businesses across the U.S to compete with and replace several traditional monitoring tools in practice. IaaS and observability monitoring is being done comprehensively within AIOps platforms, specifically if the company’s whole IT infrastructure is in the cloud.

AIOps trends and Market size in the U.S

AI-based solutions to assist DevOps capabilities is gaining popularity in Singapore

The SME segment in Singapore is expected to grow at a 30% CAGR through 2027. It has been observed that SMEs are increasingly turning to AI-based IT operations tools to improve service quality and respond to changing customer needs in a more flexible manner. In contrast to this observation, below graph illustrates the enterprises vs. SMEs AIOps scenario in 2020 and its growth prediction in 2027. 

Enterprises vs. SMEs – AIOps trends and Market size in Singapore

Real-time analytics and Infrastructure management are the driving force in Germany 

In 2020, the real-time analytics segment in Germany amassed 35% market share followed by other segments such as application performance management, network and security management, and infrastructure management. Since then, AI is increasingly being used by businesses around the country to make insightful decisions rather than depending on human interventions, and to empower IT teams to take immediate action. The below graph denotes, by the year 2027 real-time analytics and infrastructure management sharing the equal market share.

AIOps trends, applications and market size in the U.S

BFSI sector is expected to boost the AIOps market growth

The rapid growth and requirement of AIOps applications in the BFSI sector is one of the primary factors for the global AIOps market’s rise. As part of banking operations, employees, clients, and external agencies engage in a variety of regular and irregular activities and transactions. These operations must be closely monitored due to their complexity and confrontations. Considering these developments, AIOps is expected to boost market growth in 2022 and ahead while offering wide verity of applications to monitor real-time data and automated issue resolution.

Top 4 considerations for I&O leaders for successful AIOps implementation 

I&O leaders must help their organizations become more agile and responsive to the needs of business units. The right operations services, such as monitoring and metrics, can help you continuously improve your systems’ performance and responsiveness. It should start with focusing on the development of a “change engine” within the organization, using automation to reduce manual tasks, errors and inconsistencies and leveraging technology to meet compliance requirements. Thus, these I&O leaders should focus on:

• Invest and use an incremental strategy to replace rule-based event analytics and expand into domain-centric workflows like application and network diagnostics, putting practical outcomes ahead of ambitious goals
• Allow the use case to define whether AIOps should be domain-centric or domain-agnostic. To start with use specialized use case, leverage domain-centric AIOps features built into a monitoring tool, and deploy a domain-agnostic stand-alone solution with a roadmap encompassing multiple use cases
• You need to select an AIOps platform that offers bidirectional interaction with ITSM tools to enable task automation, knowledge management, and change analysis. Ensure you’re not choosing the tools that are limited to basic search-and-display functionality

• Enable continuous insights across ITOM by following the methodology of observe, think, react and learn which is discussed in the earlier section of this article

Summing Up!

AIOps will result in a sweeping change in how AI-driven IT operations are managed. Even though the primary objective for AIOps adoption is to address operational issues arising from a highly complicated IT ecosystem and keep up with a fast-paced business environment, the ability of humans to understand and trust its insights, decisions, recommendations, and predictions will be extremely crucial.

The above photo is not created by a specialized app or photoshop. It was generated by a Deep learning algorithm which uses convolutional networks to learn artistic features from various paintings and changes any photo depicting how an artist would have painted it.

Convolutional Neural Networks has become part of every state of the art solutions in areas like

• Image recognition
• Object Recognition
• Self-driving cars in identifying pedestrians, objects.
• Emotion recognition.
• Natural Language Processing.

A few days back Google surprised me with a video called Smiles 2016 where all the photos of 2016 where I was partying with family, friends, colleagues are put together. It was a collection of photos where everyone in the photo was smiling. Emotion recognition. We will discuss a couple of Deep learning architectures that powers these applications in this blog.

Before we dive into CNN lets try to understand why not Feed Forward Neural network. According to universality theorem which we discussed in the previous blog, any network will be able to approximate a function just by adding Neurons(Functions), but there are no guarantees in time when will it reach the optimal solution. Feed Forward neural networks tend to flatten images to a flat vector thus losing all the spatial information that comes with an Image. So for problems where spatial feature importance is high CNN tend to achieve higher accuracy in a very shorter time compared to Feed-Forward Neural Networks.

Before we dive into what a Convolutional Neural Network is letting get comfortable with nuts and bolts which form it.

Images

Before we dive into CNN lets take a look at how a computer looks at an image.

Wow, it’s great to know that computer sees images, videos as a matrix of numbers. A common way of looking at an image in computer vision is a matrix of dimensions Width * Height * Channels. Where Channels are Red, Green, Blue and sometimes alpha is also part of channels.

Filters

Filters are a small matrix of numbers usually of size 3*3*3 (width, height, channel) or 7*7*3. Filters perform various operations like blur, sharpen, outline on a given image. Historically these filters are carefully hand picked to gain various features of an image. In our case, CNN creates these filters automatically using a combination of techniques like Gradient descent and Backpropagation. Filters are moved across an image starting from top left to the bottom right to capture all the essential features. They are also called as kernels in Neural networks.

Convolutional

In a convolutional layer, we convolve the filter with patches across an image. For example on the left-hand side of the below image is a matrix representation of a dummy image and the middle layer is the filter or kernel. The right side of the image has the output of convolution layer. Look at the formula in the image to understand how the kernel and a part of the image are combined together to form a new pixel.

Let’s see another example of how the next pixel in the image is being generated.

Max-Pooling

Max pooling is used for reducing dimensionality and down-sampling an input. The best way to understand Max-pooling is an example. The below image describes what a 2*2 Max pooling layer does.

In both the examples for convolution and Max-pooling, the image shows for only 2 pixels, but in reality, the same technique is applied to the entire image.

Now with an understanding of all the important components, let’s take a look at how the Convolutional Neural Network looks like.

As you can see from the above image, a CNN is a combination of layers stacked together. The above architecture can be simply depicted as CONV-RELU-CONV-RELU_POOL * 3 + Fully connected Layer.

Challenges

Convolutional Neural Networks need huge amounts of labeled data and lots of computation power to get trained. They typically take weeks to get trained to achieve state of the art performance. Most of these architectures like AlexNet, ZF Net, VGG Net, Google Net, Microsoft Res Net take weeks to get trained. Does that mean, an organization without huge volumes of data and computation power cannot take advantage of it? The answer is No.

Transfer Learning to the Rescue

Most of the winners of the ILSVRC (ImageNet Large-Scale Visual Recognition Challenge) competition has open sourced the architecture and the weights associated with these networks. It turns out, most of the weights particularly that of the filters can be reused after fine tuning to the domain specific problems. So for us to take advantage of these convolutional neural networks, all we need to do is pre-train the last few layers of the network. Which in general takes very little data and computation powers. For several of our scenarios, we were able to train models with state of art performance on GPU machines in few minutes to hours.

Conclusion

Apart from use cases like image recognition , CNN is being widely used in various network topology’s like object recognition (What objects are located in images), GAN (A recent breakthrough in helping computers create realistic images), converting low resolution images to high resolution images , in revolutionizing health sector in various forms of cancer detection and many more. In recent months there were architectures built for NLP achieving state of art results.

Background

Forecasting demand for new product launches has been a major challenge for industries and cost of error has been high. Under predict demand and you lose on potential sales, overpredict them and there is excess inventory to take care of. Multiple research suggests that new product contributes to one-third of the organization sales across the various industry. Industries like Apparel Retailer or Gaming thrive on new launches and innovation, and this number can easily inflate to as high as 70%. Hence accuracy of demand forecasts has been a top priority for marketers and inventory planning teams.

There are a whole lot of analytics techniques adopted by analysts and decision scientists to better forecast potential demand, the popular ones being:

• Market Test Methods – Delphi/Survey based exercise
• Diffusion modeling
• Conjoint & Regression based look alike models

While Market Test Methods are still popular but they need a lot of domain expertise and cost intensive processes to drive desired results. In recent times, techniques like Conjoint and Regression based methods are more frequently leveraged by marketers and data scientists. A typical demand forecasting process for the same is highlighted below:

Though the process implements an analytical temper of quantifying cross synergies between business drivers and is scalable enough to generate dynamic business scenarios on the go, it falls short of expectations on following two aspects

• It includes heuristics exercise of identifying analogous products by manually defining product similarity. Besides, the robustness of this exercise is influenced by domain expertise. The manual process coupled with subjectivity of the process might lead to questionable accuracy standards.
• It is still a supervised model and the key demand drivers need manual tuning to generate better forecasting accuracy.

For retailers and manufacturers esp. apparel, food, etc. where the rate of innovation is high and assortments keep refreshing from season to season, a heuristic method would lead to high cost and error for any demand forecasting exercise.

With the advent of Deep Learning’s Image processing capabilities, the heuristic method of identifying feature similarity can be automated with a high degree of accuracy through techniques like Convoluted Neural Network (CNN). It also minimizes the need for domain expertise as it self-learns feature similarity without much supervision. Since the primary reason for including product features in demand forecasting model is to understand the cognitive influence on customer purchase behavior, a deep learning based approach can capture the same with much higher accuracies. Besides techniques like Recurrent Neural Network (RNN) can be employed to make the models better at adaptive learning and hence making the system self-reliant with negligible manual interventions.

“Since the primary reason of including product features in demand forecasting model is to understand cognitive influence on customer purchase behavior, a deep learning framework is a better and accurate approach to capture the same”

In practice, CNN and RNN are two distinct methodologies and this article highlights a case where various Deep Learning models were combined to develop a self-learning demand forecasting framework.

Case Background

An apparel retailer wanted to forecast demand for its newly launched “Footwear” styles across various lifecycle stages. The current forecasting engine implemented various supervised techniques which were ensemble to generate desired demand forecasting. It had 2 major shortcomings:

• The analogous product selection mechanism was heuristic and lead to low accuracy level in downstream processes.
• The heuristic exercise was a significant road block in evolving the current process to a scalable architecture, making the overall experience a cost intensive one.
• The engine was not able to replicate the product life cycle accurately.

Proposed Solution

We proposed to tackle the problem through an intelligent, automated and scalable framework

• Leverage Convoluted Neural Networks(CNN) to facilitate the process of identifying the analogous product. CNN techniques have been proven to generate high accuracies in image matching problems.
• Leverage Recurrent Neural Networks (RNN) to better replicate product lifecycle stages. Since RNN memory layers are better predictors of next likely event, it is an apt tool to evaluate upcoming time-based performances.
• Since the objective was to devise a scalable method, a cloud-ready easy to use UI was proposed, where user can upload the image of an upcoming style and the demand forecasts would be generated instantly.

Overall Approach

The entire framework was developed in Python using Deep Learning platforms like Tensor Flow with an interactive user interface powered by Django. The Deep Learning systems were supported through NVIDIA GPUs hosted on Google Cloud.

The demand prediction framework consists of following components to ensure an end to end analytical implementation and consumption.

1. Product Similarity Engine

An image classification algorithm was developed by leveraging Deep Learning techniques like Convolution Neural Networks. The process included:

Data Collation

• Developed an Image bank consisting of multi-style shoes across all categories/sub-categories e.g. sports, fashion, formals etc.
• Included multiple alignments of the shoe images.

Data Cleaning and Standardization

• Removed duplicate images.
• Standardized the image to a desired format and size.

Define High-Level Features

• Few key features were defined like brands, sub-category, shoe design – color, heel etc.

Image Matching Outcomes

• Implemented a CNN model with 5+ hidden layers.

The following image is an illustrative representation of the CNN architecture implemented

• Input Image: holds raw pixel values of the image with features being width, height & RGB values.
• Convolution: Conv Net is to extract features from input data. Formation of matrix by sliding filters over an image and computing dot product is called “Feature Map”.
• Non-Linearity – RELU: This layer applies element-wise activation filter leveraged to stimulate non-linearity relationships in a standard ANN.
• Pooling:  Reduces the dimensionality of each feature map and retains important information. Helps in arriving at a scale invariant representation of an image.
• Dropouts: To prevent overfitting random connections are severed.
• SoftMax Layer: Output layer that classifies the image to appropriate category/subcategory/heel height classes.

Identified Top N matching shoes and calculated their probability scores. Classified image orientation as top, side (right/left) alignment of the same image: 

Similarity Index- Calculated based on the normalized overall probability scores.

Analogous Product: Attribute Similarity Snapshot (Sample Attributes Highlighted)

2. Forecasting Engine

A demand forecasting engine was developed on the available data by evaluating various factors like:

• Promotions – Discounts, Markdown
• Pricing changes
• Seasonality – Holiday sales
• Average customer rating
• Product Attributes – This was sourced from the CNN exercise highlighted in the previous step
• Product Lifecycle – High sales in initial weeks followed by declining trend

The following image is an illustrative representation of the demand forecasting model based on RNN architecture.

The RNN implementation was done using Keras Sequential model and the loss function was estimated using “mean squared error” method.

Demand Forecast Outcome

The accuracy from the proposed Deep Learning framework was in the range of 85-90% which was an improvement on the existing methodology of 60-65%.

Web UI for Analytical Consumption

An illustrative snapshot is highlighted below:

Benefits and Impact

• Higher accuracy through better learning of the product lifecycle.
• The overall process is self-learning and hence can be scaled quickly.
• Automation of decision intensive processes like analogous product selection led to reduction in execution time.
• Long-term cost benefits are higher.

Key Challenges & Opportunities

• The image matching process requires huge data to train.
• The feature selection method can be an automated through unsupervised techniques like Deep Auto Encoders which will further improve scalability.
• Managing image data is a cost intensive process but it can be rationalized over time.
• The process accuracies can be improved by creating a deeper architecture of the network and an additional one-time investment of GPU configurations.

Abstract

This article edition of Bayesian Analysis with Python introduced some basic concepts applied to the Bayesian Inference along with some practical implementations in Python using PyMC3, a state-of-the-art open-source probabilistic programming framework for exploratory analysis of the Bayesian models.

The main concepts of Bayesian statistics are covered using a practical and computational approach. The article covers the main concepts of Bayesian and Frequentist approaches, Naive Bayes algorithm and its assumptions, challenges of computational intractability in high dimensional data and approximation, sampling techniques to overcome challenges, etc. The results of Bayesian Linear Regressions are inferred and discussed for the brevity of concepts.

Introduction

Frequentist vs. Bayesian approaches for inferential statistics are interesting viewpoints worth exploring. Given the task at hand, it is always better to understand the applicability, advantages, and limitations of the available approaches.

In this article, we will be focusing on explaining the idea of Bayesian modeling and its difference from the frequentist counterpart. To make the discussion a little bit more intriguing and informative, these concepts are explained with a Bayesian Linear Regression (BLR) model and a Frequentist Linear Regression (LR) model.

Bayesian and Frequentist Approaches

The Bayesian Approach:

Bayesian approach is based on the idea that, given the data and a probabilistic model (which we assume can model the data well), we can find out the posterior distribution of the model’s parameters. For e.g.

In Bayesian Linear Regression approach, not only the dependent variable y,  but also the parameters(β) are assumed to be drawn from a probability distribution, such as Gaussian distribution with mean=β^TX, and variance =σ²I (refer equation 1). The outputs of BLR is a distribution, which can be used for inferring new data points.

The Frequentist Approach, on the other hand, is based on the idea that given the data, the model and the model parameters, we can use this model to infer new data. This is commonly known as the Linear Regression Approach. In LR approach, the dependent variable (y) is a linear combination of weights term-times the independent variable (x), and e is the error term due to the random noise.

Ordinary Least Square (OLS) is the method of estimating the unknown parameters of LR model. In OLS method, the parameters which minimize the sum of squared errors of training data are chosen. The output of OLS are “single point” estimates for the best model parameter.

Let’s get started with Naive Bayes Algorithm, which is the backbone of Bayesian machine learning algorithms. Here, we can predict only one value of y, so basically it is a point estimation

Naive Bayes Algorithm for Classification       

Discussions on Bayesian Machine Learning models require a thorough understanding of probability concepts and the Bayes Theorem. So, now we discuss Bayes’ Algorithm. Bayes’ theorem finds the probability of an event occurring, given the probability of an already occurred event. Suppose we have a dataset with 7 features/attributes/independent variables (x₁, x_2, x₃,…, x₇), we call this data tuple as X. Assume H is the hypothesis of the tuple belonging to class C. In Bayesian terminology, it is known as the evidence. y is the dependent variable/response variable (i.e., the class in classification problem). Then Mathematically, Bayes theorem is stated as :

Where:  

1. P(H|X) is the probability that the hypothesis H holds correct, given that we know the ‘evidence’ or attribute description of X. P(H|X) is the probability of H conditioned on X, a.k.a., Posterior Probability.                              
2. P(X|H) is the posterior probability of X conditioned on H and is also known as ‘Likelihood’.
3. P(H) is the prior probability of H. This is the fraction of occurrences for each class out of total number of samples.
4. P(X) is the prior probability of evidence (data tuple X), described by measurements made on a set of attributes (x₁, x_2, x₃,…, x₇).

As we can see, the posterior probability of H conditioned on X is directly proportional to likelihood times prior probability of class and is inversely proportional to the ‘Evidence’.

Bayesian approach for regression problem: Assumptions of Bayes theorem, given a sales prediction problem with 7 independent variables.

i) Each pair of features in the dataset are independent of each other. For e.g., feature x₁ has no effect on x₂, & x₂ has no effect on feature x₇.
ii) Each feature makes an equal contribution towards the dependent variable.

Finding the posterior distribution of model parameters is computationally intractable for continuous variables, we use Markov Chain Monte Carlo and Variational Inferencing methods to overcome this issue.

From Naive Bayes theorem (equation 3), posterior calculation needs a prior, a likelihood and evidence. Prior and likelihood are calculated easily as they are defined by the assumed model. As P(X) doesn’t depend on H and given the values of features, the denominator is constant. So, P(X) is just a normalization constant. We need to maximize the value of numerator in equation 3. However, the evidence (probability of data) is calculated as:

Calculating the integral is computationally intractable with high dimensional data. In order to build faster and scalable systems, we require some sampling or approximation techniques to calculate the posterior distribution of parameters given in the observed data. In this section, two important methods for approximating intractable computations are discussed. These are sampling-based approach. Markov-chain Monte Carlo Sampling (MCMC sampling) and approximation-based approach known as Variational Inferencing (VI). Brief introduction of these techniques are as mentioned below:

• MCMC– We use sampling techniques like MCMC to draw samples from the distribution, followed by approximating the distribution of the posterior. Refer to George’s blog [1], for more details on MCMC initialization, sampling and trace diagnostics.
• VI– Variational Inferencing method tries to find the best approximation of the distribution from a parameter family. It uses an optimization process over parameters to find the best approximation. In PyMC3, we can use Automatic Differentiation Variational Inference (ADVI), which tries to minimize the Kullback–Leibler (KL) divergence between a given parameter family distribution and the distribution proposed by the VI method.

Prior Selection: Where is the prior in data, from where do I get one?

Bayesian modelling gives alternatives to include prior information into the modelling process. If we have domain knowledge or an intelligent guess about the weight values of independent variables, we can make use of this prior information. This is unlike the frequentist approach, which assumes that the weight values of independent variables come from the data itself. According to Bayes theorem:

Now that the method for finding posterior distribution of model parameters are being discussed, the next obvious question based on equation 5 is how to find a good prior. Refer [2] for understanding how to select a good prior for the problem statement. Broadly speaking, the information contained in the prior has a direct impact on the posterior calculations. If we have a more “revealing prior” (a.k.a., a strong belief about the parameters), we need more data to “alter” this belief. The posterior is mostly driven by prior. Similarly, if we have an “vague prior” (a.k.a., no information about the distribution of parameters), the posterior is much driven by data. It means that if we have a lot of data, the likelihood will wash away the prior assumptions [3]. In BLR, the prior knowledge modelled by a probability distribution is updated with every new sample (which is modelled by some other probability distribution).

Modelling Using PyMC3 Library for Bayesian Inferencing

Following snippets of code (borrowed from [4]), shows Bayesian Linear model initialization using PyMC3 python package. PyMC3 model is initialized using “with pm.Model()” statement. The variables are assumed to follow a Gaussian distribution and Generalized Linear Models (GLMs) used for modelling.  For an in-depth understanding on PyMc3 library, I recommend Davidson-Pilon’s book [5] on Bayesian methods.

Fig. 1 Traceplot shows the posterior distribution for the model parameters as shown on the left hand side. The progression of the samples drawn in the trace for variables are shown on the right hand side.

We can use “Traceplot” to show the posterior distribution for the model parameters and shown on the left-hand side of Fig. 1. The samples drawn in the trace for the independent variables and the intercept for 1,000 iterations are shown on the right-hand side of the Fig 1. Two colours – orange and blue, represent the two Markov chains.

After convergence, we get the coefficients of each feature, which is its effectiveness in explaining the dependent variable. The values represented in red are the Maximum a posteriori estimate (MAP), which is the mean of the variable value from the distribution. The sales can be predicted using the formula:

As it is a Bayesian approach, the model parameters are distributions. Following plots show the posterior distribution in the form of histogram. Here the variables show 94% HPD (Highest Posterior Density). HPD in Bayesian statistics is the credible interval, which tells us we are 94% sure that the parameter of interest falls in the given interval (for variable x₆, the value range is -0.023 to 0.36).

We can see that the posteriors are spread out, which is an indicative of less data points used for modelling, and the range of values each independent variable can take is not modelled within a small range (uncertainty in parameter values are very high). For e.g., for variable x₆, the value range is from -0.023 to 0.36, and the mean is 0.17. As we add more data, the Bayesian model can shrink this range to a smaller interval, resulting in more accurate values for weights parameters.

When to use linear and BLR, Map, etc. Do we go Bayesian or Frequentist?

The equation for linear regression on the same dataset is obtained as:

If we see Linear regression equation (eq. 7) and Bayesian Linear regression equation (eq. 6), there is a slight change in the weight’s values. So, which approach should we take up? Bayesian or Frequentist, given that both are yielding approximately the same results?

When we have a prior belief about the distributions of the weight variables (without seeing the data) and want this information to be included into the modelling process, followed by automatic belief adaptation as we gather more data, Bayesian is a preferable approach. If we don’t want to include any prior belief and model adaptions, the weight variables as point estimates, go for Linear regression. Why are the results of both models approximately the same? 

The maximum a posteriori estimates (MAP) for each variable is the peak value of the variable in the distribution (shown in Fig.2)  close to the point estimates for variables in LR model. This is the theoretical explanation for real-world problems. Try using both approaches, as the performance can vary widely based on the number of data points, and data characteristics.

Conclusion

This blog is an attempt to discuss the concepts of Bayesian inferencing and its implementation using PyMC3. It started off with the decade’s old Frequentist-Bayesian perspective and moved on to the backbone of Bayesian modelling, which is Bayes theorem. Once setting the foundations, the concepts of intractability to evaluate posterior distributions of continuous variables along with the solutions via sampling methods viz., MCMC and VI are discussed.  A strong connection between the posterior, prior and likelihood is discussed, taking into consideration the data available in hand. Next, the Bayesian linear regression modelling using PyMc3 is discussed, along with the interpretations of results and graphs. Lastly, we discussed why and when to use Bayesian linear regression.

Resources:

The following are the resources to get started with Bayesian inferencing using PyMC3.

[1] https://eigenfoo.xyz/bayesian-modelling-cookbook/

[2] https://github.com/stan-dev/stan/wiki/Prior-Choice-Recommendations

[3] https://stats.stackexchange.com/questions/58564/help-me-understand-bayesian-prior-and-posterior-distributions

[4] https://towardsdatascience.com/bayesian-linear-regression-in-python-using-machine-learning-to-predict-student-grades-part-2-b72059a8ac7e

[5] Davidson-Pilon, Cameron. Bayesian methods for hackers: probabilistic programming and Bayesian inference. Addison-Wesley Professional, 2015