machine learning Archives - DexLab Analytics | Big Data Hadoop SAS R Analytics Predictive Modeling & Excel VBA

Complete study of COVID-19 in India (Part II) – Laboratory and Testing

Posted on May 13, 2020May 23, 2020 by Dexlab

The first case of the 2019-2020 Coronavirus pandemic in India was reported on January 30, 2020, originating in China. Experts suggest the number of infections could be much higher as India’s testing rates are among the lowest in the world. The infection rate of COVID-19 in India is 1.7, significantly lower than in the worst affected countries.

The World Health Organisation chief executive director of health emergencies program Michael Ryan said that India has “tremendous capacity” to deal with the coronavirus outbreak, and as the second most populous country, will have enormous impact on the world’s ability to deal with it.

DexLab Analytics, in the first part of this blog series, studied the statewise breakup of COVID-19 cases in India through a Jupyter Notebook. Libraries were called, maps were drawnand data was taken from Kaggle.

The data and code sheet can be found below.

In this part of the blog series we will study how states are performing with regard to laboratories and testing. First we make three data sets – that of confirmed cases, recovered cases and cases of deaths.

We first plot this data on a graph and study it carefully. Then we make a pivot table and study the data. We then also study which state is performing how many tests on people. Kerala is found to have done the maximum number of tests (Fig.1.).

Fig. 1.

The purpose of this video is to teach you how to use visual graphs in Python. Now we aim to find why testing is underdone in states. Is there a possibility of a lesser number of labs in the first place? We get a graph (Fig. 2.) that shows us how many labs each state has for testing COVID-19 samples.

Fig. 2.

For the complete study watch the video attached herewith. This study was brought to you by DexLab Analytics. DexLab Analytics is a premiere Artificial Intelligence training institute in Gurgaon.

Complete Statewise Study on COVID-19 in India (Part I)

Posted on May 11, 2020July 7, 2020 by Dexlab

The first case of the 2019-2020 Coronavirus pandemic in India was reported on January 30, 2020, originating in China. Experts suggest the number of infections could be much higher as India’s testing rates are among the lowest in the world. The infection rate of COVID-19 in India is 1.7, significantly lower than in the worst affected countries.

The World Health Organisation chief executive director of health emergencies program Michael Ryan said that India has “tremendous capacity” to deal with the coronavirus outbreak, and as the second most populous country, will have enormous impact on the world’s ability to deal with it.

Other commentators worried about the economic devastation caused by the lockdown that has huge effects on informal workers, micro and small enterprises and farmers and self employed people who are left without a livelihood in the absence of transportation and access to markets.

The lockdown was justified by the government and other agencies for being pre-emptive to prevent India from entering a higher stage which could make handling very difficult and cause even more losses thereafter. According to a study by Shiv Nadar University, India could have witnesses a surge of 31,000 cases between March 24 and April 14 without lockdown.

So we call a Jupyter Notebook in Python to study India’s COVID-19 story.

The data and code sheet used in this study can be found below.

We will first import all libraries like pandas and numpy. All the data has been taken from kaggle. We then take the data and work a dataframe on it. And then we generate an India map to study the spread of SARS-CoV-2.

Fig. 1.

For more on this, please watch the complete video attached herewith. This study was brought to you by DexLab Analytics. DexLab Analytics is a premiere Artificial Intelligence training institute in Gurgaon.

Netflix develops in own data science management tool and open sources it

Posted on January 28, 2020May 23, 2020 by Dexlab

Netflix in December last year introduced its own python framework called Metaflow. It was developed to apply to data science with a vision to make scalability a seamless proposition. Metaflow’s biggest strength is that it makes running the pipeline (constructed as a series of steps in a graph) easily movable from a stationary machine to cloud platforms (currently only the Amazon Web Services (AWS)).

What does Metaflow really do? Well, it primarily “provides a layer of abstraction” on computing resources. What it translates to is the fact that a programmer can concentrate on writing/working code while Metaflow will handle the aspect which ensures the code runs on machines.

Metaflow manages and oversees Python data science projects addressing the entire data science workflow (from prototype to model deployment), works with various machine learning libraries and amalgamates with AWS.

Machine learning and data science projects require systems to follow and track the trajectory and development of the code, data, and models. Doing this task manually is prone to mistakes and errors. Moreover, source code management tools like Git are not at all well-suited to doing these tasks.

Metaflow provides Python Application Programming Interfaces (APIs) to the entire stack of technologies in a data science workflow, from access to the data, versioning, model training, scheduling, and model deployment, says a report.

Netflix built Metaflow to provide its own data scientists and developers with “a unified API to the infrastructure stack that is required to execute data science projects, from prototype to production,” and to “focus on the widest variety of ML use cases, many of which are small or medium-sized, which many companies face on a day to day basis”, Metaflow’s introductory documentation says.

Metaflow is not biased. It does not favor any one machine learning framework or data science library over another. The video-streaming giant deploys machine learning across all aspects of its business, from screenplay analysis, to optimizing production schedules and pricing. It is bent on using Python to the best limits the programming language can stretch. For the best Data Science Courses in Gurgaon or Python training institute in Delhi, you can check out the Dexlab Analytics courses online.

Interested in a career in Data Analyst?
To learn more about Data Analyst with Advanced excel course – Enrol Now.
To learn more about Data Analyst with R Course – Enrol Now.
To learn more about Big Data Course – Enrol Now.
To learn more about Machine Learning Using Python and Spark – Enrol Now.
To learn more about Data Analyst with SAS Course – Enrol Now.
To learn more about Data Analyst with Apache Spark Course – Enrol Now.
To learn more about Data Analyst with Market Risk Analytics and Modelling Course – Enrol Now.

Skills required during Interviews for a Data Scientist @ Facebook, Intel, Ebay. Square etc.

Posted on June 9, 2017May 23, 2020 by Dexlab

Basic Programming Languages: You should know a statistical programming language, like R or Python (along with Numpy and Pandas Libraries), and a database querying language like SQL

Statistics: You should be able to explain phrases like null hypothesis, P-value, maximum likelihood estimators and confidence intervals. Statistics is important to crunch data and to pick out the most important figures out of a huge dataset. This is critical in the decision-making process and to design experiments.

Machine Learning: You should be able to explain K-nearest neighbors, random forests, and ensemble methods. These techniques typically are implemented in R or Python. These algorithms show to employers that you have exposure to how data science can be used in more practical manners.

Data Wrangling: You should be able to clean up data. This basically means understanding that “California” and “CA” are the same thing – a negative number cannot exist in a dataset that describes population. It is all about identifying corrupt (or impure) data and and correcting/deleting them.

Data Visualization: Data scientist is useless on his or her own. They need to communicate their findings to Product Managers in order to make sure those data are manifesting into real applications. Thus, familiarity with data visualization tools like ggplot is very important (so you can SHOW data, not just talk about them)

Software Engineering: You should know algorithms and data structures, as they are often necessary in creating efficient algorithms for machine learning. Know the use cases and run time of these data structures: Queues, Arrays, Lists, Stacks, Trees, etc.

What they look for? @ Mu-Sigma, Fractal Analytics

- Most of the analytics and data science companies, including third party analytics companies such as Mu-sigma and Fractal hire fresher’s in big numbers (some time in hundreds every year).
- You see one of the main reasons why they are able to survive in this industry is the “Cost Arbitrage” benefit between the US and other developed countries vs India.
- Generally speaking, they normally pay significantly lower for India talent in India compared to the same talent in the USA. Furthermore, hiring fresh talent from the campuses is one of the key strategies for them to maintain the low cost structure.
- If they are visiting your campuses for interview process, you should apply. In case if they are not visiting your campus, drop your resume to them using their corporate email id that you can find on their websites.
- Better will be to find someone in your network (such as seniors) who are working for these companies and ask them to refer you. This is normally the most effective approach after the campus placements.

Key Skills that look for are-

Love for numbers and quantitative stuff
Grit to keep on learning
Some programming experience (preferred)
Structured thinking approach
Passion for solving problems
Willingness to learn statistical concepts

Technical Skills

Math (e.g. linear algebra, calculus and probability)
Statistics (e.g. hypothesis testing and summary statistics)
Machine learning tools and techniques (e.g. k-nearest neighbors, random forests, ensemble methods, etc.)
Software engineering skills (e.g. distributed computing, algorithms and data structures)
Data mining
Data cleaning and munging
Data visualization (e.g. ggplot and d3.js) and reporting techniques
Unstructured data techniques
Python / R and/or SAS languages
SQL databases and database querying languages
Python (most common), C/C++ Java, Perl
Big data platforms like Hadoop, Hive & Pig

Business Skills

Analytic Problem-Solving: Approaching high-level challenges with a clear eye on what is important; employing the right approach/methods to make the maximum use of time and human resources.
Effective Communication: Detailing your techniques and discoveries to technical and non-technical audiences in a language they can understand.
Intellectual Curiosity: Exploring new territories and finding creative and unusual ways to solve problems.
Industry Knowledge: Understanding the way your chosen industryfunctions and how data are collected, analyzed and utilized.

Interested in a career in Data Analyst?
To learn more about Data Analyst with Advanced excel course – Enrol Now.
To learn more about Data Analyst with R Course – Enrol Now.
To learn more about Big Data Course – Enrol Now.
To learn more about Machine Learning Using Python and Spark – Enrol Now.
To learn more about Data Analyst with SAS Course – Enrol Now.
To learn more about Data Analyst with Apache Spark Course – Enrol Now.
To learn more about Data Analyst with Market Risk Analytics and Modelling Course – Enrol Now.

How Predictive Analysis Could Have Saved the World from Ransomware

Posted on May 16, 2017May 18, 2017 by Dexlab

Kudos to you, if you have stayed offline for the last couple of days, so you could actually spend the weekend well with your family and loved ones. The world is reeling under the shattering news surrounding WannaCry Ransomware this weekend. The situation was worse on Monday, after the offices opened. Going by the figures, revealed out on Monday evening by Elliptic, a Bitcoin forensics firm, which is keeping a watch overall – $57,282.23 in ransom has been shelled out to the hackers of Ransomware malware attack, who took over hundreds and thousands of computers worldwide on Friday and through the weekend.

Continue reading “How Predictive Analysis Could Have Saved the World from Ransomware”

Will AI Replace The Intelligentsia? Google’s AI writes mournful poetry

Posted on October 4, 2016November 14, 2018 by Dexlab

It is no new news that Artificial Intelligence can now control self driving cars; they can beat the best humans at highly challenging board games like chess, and even fight cancer. But still one thing it cannot do perfectly is communicate.

So, to help solve this problem Google has been feeding its Artificial Intelligence with more than 11,000 unpublished books, which include more than 3000 steamy romantic titles. And in response the AI has penned down its own version of mournful poems.

The poems read something like this:

I went o the store to buy some groceries.

I store to buy some groceries.

I were to buy any groceries.

Horses are to buy any groceries.

Horses are to buy any animal.

Horses the favourite any animal.

Horses the favourite favourite animal.

Horses are my favourite animal.

And here is another one from Google’s AI:

he said.

“no,” he said.

“no,” i said.

“i know,” she said.

“thank you,” she said.

“come with me,” she said.

“talk to me,” she said.

“don’t worry about it,” she said.

The way this happened was, Google’s team fed their AI with unpublished works into a neural network and gave the system two sentences from the book; it was then up to this ingenious artificial intelligence to build its own poetry based on available information.

In example above, the team of researchers gave their AI two sentences one about buying some groceries and the other one about horses being a favourite animal (these are the first and the last lines of the above mentioned passages). The team then directed the artificial intelligence to morph between the two sentences.

In the research paper the team further went on to explain the AI system was able to “create coherent and diverse sentences through purely continuous sampling”.

With the use of an autoanecdoter, which is a type of AI network that makes use of data sets to reproduce a result, in this case that was writing sentences, using much fewer steps the team was able to produce these sentences.

The main principle behind this research is to create an Artificial Intelligence which will be proficient in communicating via “natural language sentences”.

This research holds the possibilities of developing a system that is capable of communicating in a more human-like manner. Such a breakthrough is essential in the creation of more useful and responsive chat bots and Artificial Intelligence powered personal assistants like that of Siri and Google Now.

In a similar project, the researchers at Google have been teaching an AI how to understand language by replicating and predicting the work of bygone authors and poets under their project Gutenberg.

This standalone team at Google fed the AI with an input sentence and then asked it to predict what should come next. And by analysing the text, the AI was capable of identifying what author was likely to have written the sentence and was able to emulate his style.

In another incident, on June, 2015 another team of talented researchers at Google were able to create a chatbot that even threatened its creators. The AI learned the art of conversation by analysis of a million movie scripts thereby allowing it to realize and muse on the meaning of life, the colour of blood, and even on deeper subjects like mortality; Ss, much so that the bot could even get angry on its human inquisitor. When the bot was asked with a puzzling philosophical question about what is the meaning of life, it replied by saying – “to live forever”.

In other such similar works, Facebook has also been teaching its artificial intelligence with the use of children’s books. As per the New Scientist which is a social network, it has been using novels such as The Jungle Book, Alice in Wonderland and Peter Pan.

If you are yearning for some more of AI’s written word, then here are the rest of Google AI’s poems.

You’re right.

“All right.

You’re right.

Okay, fine.

“Okay, fine.

Yes, right here.

No, not right now.

“No, not right now.

“Talk to me right now.

Please talk to me right now.

I’ll talk to you right now.

“I’ll talk to you right now.

“You need to talk to me now. —

Amazing, isn’t it?

So, what is it?

It hurts, isn’t it?

Why would you do that?

“You can do it.

“I can do it.

I can’t do it.

“I can do it.

“Don’t do it.

“I can do it.

I couldn’t do it. —

There is no one else in the world.

There is no one else in sight.

They were the only ones who mattered.

They were the only ones left.

He had to be with me.

She had to be with him.

I had to do this.

I wanted to kill him.

I started to cry.

I turned to him. —

I don’t like it, he said.

I waited for what had happened.

It was almost thirty years ago.

It was over thirty years ago.

That was six years ago.

He had died two years ago.

Ten, thirty years ago. — “it’s all right here.

“Everything is all right here.

“It’s all right here.

It’s all right here.

We are all right here.

Come here in five minutes.

“But you need to talk to me now.

To feed in adequate information on Machine Learning Using Python, reach us at DexLab Analytics. Our Machine Learning Certification is garnering a lot of attention owing to its program-centric course module.

Interested in a career in Data Analyst?
To learn more about Data Analyst with Advanced excel course – Enrol Now.
To learn more about Data Analyst with R Course – Enrol Now.
To learn more about Big Data Course – Enrol Now.
To learn more about Machine Learning Using Python and Spark – Enrol Now.
To learn more about Data Analyst with SAS Course – Enrol Now.
To learn more about Data Analyst with Apache Spark Course – Enrol Now.
To learn more about Data Analyst with Market Risk Analytics and Modelling Course – Enrol Now.

Our Machine Learning Training Demo Session Was a Raging Success

Posted on September 21, 2016September 28, 2016 by Dexlab

If you follow our social media activities at DexLab Analytics, then you must be aware of our recent event held last Sunday, on 18^th September, 2016. We had welcomed people from all around the globe to join us at our Machine Learning training demo session which was completely free to join.

We are glad to announce that we have received a huge response from professionals around the world who made our event a complete full house! The total number of participants in the event was more than 50 people who had logged in from their own isolated remote locations to hear our instructor’s take on Machine Learning. Continue reading “Our Machine Learning Training Demo Session Was a Raging Success”

Know The Answer To These Interview Questions To Get A Job As Data Analyst

Posted on September 9, 2016May 23, 2020 by Dexlab

With this Data analyst interview guide you will know what to expect in an interview round for a position of data analyst.

A good data analyst or scientist must be capable of drawing actionable insights from the data that a company generates. They must possess a good sense of what data they must collect and should have a solid process for carrying it out effectively using processes of data analysis and building predictive models.

A data analyst must possess a strong foundation in the following topics: operations research, statistics, machine learning along with some database skills, such as SQL or SAS in order to clean, retrieve and process the data from different sources. One can lead to this role from different pathways thus candidates can expect to be bombarded with questions relevant to statistics ort mathematics and even computer programming or engineering.

Data scientists are also often required to script programs using R or Python or Matlab and the role will typically not place emphasis on the programming skills or practices and the general software engineering skills which is necessary for working with production quality software.

Here is a list of common data analyst interview questions:

Operational questions:

Describe the steps that you follow when creating a design a data-driven model to manage a business problem. For example you may try and automatically classify customer support mails, by either sentiment or topic. Another task may be to predict a company’s employee churn.
What models would you classify as simple models and which are the ones that are complex according to you? What are the comparative strengths and weaknesses of choosing a more complex model over a simplistic one?
What are the possible ways in which you can combine models to create an ensemble model and what are the main advantages of doing this?
Tell us about certain pre-processing steps that you may carry out on data before using them to train a model and describe the conditions under which they may be applied.

Role specific questions:

About basic ideas in probability, statistics and machine learning:

Define what is confidence interval and why do you think it is useful?
What is the main difference between correlation and independence?
What is Bayes Theorem? What is conditional probability? What is its use in practice?
When and how do you understand that you have collected ample data for building a model?
Tell us the difference between classification and regression.

Hope this list of common data science interview questions will prepare you for a job at a reputable data analysis company. For more such data science news, tutorials and articles with emphasis on programming and analytics view our regular updates from DexLab Analytics.

Interested in a career in Data Analyst?
To learn more about Data Analyst with Advanced excel course – Enrol Now.
To learn more about Data Analyst with R Course – Enrol Now.
To learn more about Big Data Course – Enrol Now.
To learn more about Machine Learning Using Python and Spark – Enrol Now.
To learn more about Data Analyst with SAS Course – Enrol Now.
To learn more about Data Analyst with Apache Spark Course – Enrol Now.
To learn more about Data Analyst with Market Risk Analytics and Modelling Course – Enrol Now.

The Most Important Algorithms Every Data Scientist Must Know

Posted on April 26, 2016May 23, 2020 by Dexlab

Algorithms are now like the air we breathe; it has become an inevitable part of our daily lives and is also included in all types of businesses. Experts like Gartner has called this age as the algorithm business which is the key driving force that is overthrowing the traditional ways in which we do our business and manage operations.

In fact the algorithm boom with uber diversification has reached a new high, so much so that now each function in a business has its own algorithm and one can buy their own from the algorithm marketplace. This was developed by algorithm developers at Algorithmia to save the precious time and money of business operators and other fellow developers and offers a plethora of more than 800 algorithms in the fields of machine learning, audio and visual processing and computer vision.

But we as data enthusiasts in the same field with an undying love for algorithm would like to suggest that not all the algorithms from the Algorithmia marketplace may be suitable for your needs. Business needs are highly subjective and environment based. And things as dynamic as algorithms can produce different types of results even in the slightly different situations. Also the use of algorithms depends on a number of factors on how they can be applied and what results one can expect from their application. The variables on which the application of algorithms depends are as follows: type and volume of the data sets, the function the algorithm will be applied for and the industry in which the algorithm will be applied.

Hence, not always reaching for the easy option of buying a readymade algorithm off the shelf and simply tweaking it to fit into your model may not always be the most cost-effective or time saving way to go. So, it is highly recommended for data scientists to educate themselves well on the most important algorithms that must be known by them, as well as the back of their hands. A data scientist must also know how each algorithm is developed and also which purpose calls for which algorithm to be applied.

So, our experts associated with DexLab Analytics developed an infographic to let big data analysts know the 12 most essential algorithms that must still be included in the repertoire of a skilled data scientist. To know more about data science courses drop DexLab Analytics and find your true data-based calling.

Interested in a career in Data Analyst?
To learn more about Data Analyst with Advanced excel course – Enrol Now.
To learn more about Data Analyst with R Course – Enrol Now.
To learn more about Big Data Course – Enrol Now.
To learn more about Machine Learning Using Python and Spark – Enrol Now.
To learn more about Data Analyst with SAS Course – Enrol Now.
To learn more about Data Analyst with Apache Spark Course – Enrol Now.
To learn more about Data Analyst with Market Risk Analytics and Modelling Course – Enrol Now.

Call us to know more

Gurgaon

Kolkata