Dexlab, Author at DexLab Analytics | Big Data Hadoop SAS R Analytics Predictive Modeling & Excel VBA

How Much You Can Expect To Earn As A SAS Expert in India

Posted on October 31, 2015April 11, 2016 by Dexlab

The salary, on an average, for a SAS Programmer residing in India stands at Rs. 396,305. This salary is calculated on a yearly basis. It has been observed that most SAS programmers switch to other, mainly more senior, positions after 10 years in this particular career path. This is a field that has strong emphasis on experience which is reflected in the salary as well. For a higher paying position, associated skills like MS Excel, UNIX and SQL/PL are highly recommended.
How Much You Can Expect To Earn As A SAS Expert in India

Average Salary

According to National Salary Data, the salary of a SAS Programmer might range from anywhere in between Rs 201,550 at the bottom end of the spectrum to a whopping Rs 857,227 at the top. Further, bonuses range from none to Rs 100,591. In some instances there is also an option of profit sharing which starts from none and caps at Rs 59,598. This pushes the pay range to lie anywhere in between Rs. 205,987 to Rs 917,343.

Continue reading “How Much You Can Expect To Earn As A SAS Expert in India”

Top 10 Books Essential for SAS Beginners – Part1

Posted on October 29, 2015April 11, 2016 by Dexlab

Statistical Analysis System or more popularly abbreviated simply as SAS is a suite of software tools which was created by the SAS Institute for use in business intelligence, analysis of multivariates, management of data as well as predictive analytics. The development of the SAS suite initially took place in 1966 at the “ North Carolina State University” and was maintained by the same institute till 1976 when the SAS Institute happened to be incorporated. Later on new procedures in statistics, more components and JMP were introduces as part of the SAS Bundle. A point and click UI was followed in its 9^th version realized in 2004. Social Media analytics found its pride of place in 2010.

Continue reading “Top 10 Books Essential for SAS Beginners – Part1”

HIVE – User Defined Functions

Posted on October 27, 2015October 27, 2015 by Dexlab

Though, Hive has a list of built in functions, in some scenarios we need user defined functions to be written in Java for some specific use cases.

We can use two interfaces which can be used to write UDFs for apache Hive.

The simple API (apache.hadoop.hive.ql.exec.UDF) can be used as long as our function reads and returns primitive types. Means, basic Hadoop & Hive writable types – Text, LongWritable, IntWritable and DoubleWritable etc.

If you plan to write a UDF that deals with embedded data structures, such asList, Mapand Set, then you need to useapache.hadoop.hive.ql.udf.generic.GenericUDF, which is a little more involved.

Simple API – apache.hadoop.hive.ql.exec.UDF
Complex API – apache.hadoop.hive.ql.udf.generic.GenericUDF

Steps to create Hive-UDF

Step 1:-

Open your Eclipse then create a java Class Name

Step 2:-

Add Jar files to project folder

Step 3 :-

Extend UDF Abstract Class

public class classname extends UDF and you return the value.

Step 4 :-

Implement evaluate() method . This method is called once for every row of data being processed

Step 5:-

Compile and create jar file.

Step 6:-

Add jar file to hive class path.

In hive terminal – add jar <jar file path>

Step 7 :-

Create temporary function in Hive Terminal.

CREATE temporary function Convert as ‘udf.Convert′;

udf represents the package name and Convert represents the program name .

For example:

packageudf

importorg.apache.hadoop.hive.ql.exec.UDF;

importorg.apache.hadoop.io.Text;

publicclassConvertextends UDF{

private Text result =new Text();

public Text evaluate(String str){

int number;

number=Integer.parseInt(str);

float fno=(float) number;

String res=Float.toString(fno);

result.set(res);

return result;

}

Here, We have extended UDF abstract class.

This code converts Int to Float.

Assuming a hive table Demo contains column ID with following data:

Select Convert(ID) from Demo gives following output :

1.0

2.0

3.0

5.0

Big Data and the Cloud- An Eclectic Mix

Posted on October 24, 2015November 30, 2019 by Dexlab

The FINRA or The Financial Industry Regulatory Authority, Inc. makes analysis of up to no less than 75 billion events each and every day. It is little wonder then that it finds its data center nearly filled to capacity. FINRA is looking forward to migrating to the cloud in order to continue to provide the protection for investors and continually respond to the market that it is famed for.

According to Matt Cardillo who is the Senior Director at FINRA, they are eyeing the elasticity that is enabled by cloud storage. He further continued also on their radar was an approach change in order to respond to market and volume data change along with changes in the behavior of users. Volatile markets result in usage spikes and also attract a whole lot more of users in their system.

The surveillance program undertaken by FINRA performs analysis of data for suspicious activities as well as potential fraud. Their algorithms go through and analyze the data for any abnormalities or activities that might not be normal. They have in place alerts and exceptions that take stock of situations and then have access to analytics that help to determine if there is indeed a problem or whether it is a false call.

Stay Ahead of the Big Data Curve

Almost every day a new tool emerges to take stock of Analytics in the brave new world of Big Data Tech. According to Cardillo the kudos for staying ahead of the big data curve goes to the skilled staff at FINRA. He says that his people are innovative and are only too keen to embrace the latest advancements in technology. He confesses that after reverting to the cloud some of their present tech as well as tools will become irrelevant. But they are banking big on open source especially frameworks like Hive, Hadoop and Spark to get most out of the elasticity needed by their business.

Interested in a career in Data Analyst?
To learn more about Data Analyst with Advanced excel course – Enrol Now.
To learn more about Data Analyst with R Course – Enrol Now.
To learn more about Big Data Course – Enrol Now.
To learn more about Machine Learning Using Python and Spark – Enrol Now.
To learn more about Data Analyst with SAS Course – Enrol Now.
To learn more about Data Analyst with Apache Spark Course – Enrol Now.
To learn more about Data Analyst with Market Risk Analytics and Modelling Course – Enrol Now.

Why Prefer R Programming

Posted on October 22, 2015October 19, 2015 by Dexlab

R Programming comes in pretty handy in case of machine learning as well as numeric analysis. With emergence of machines as generators of data, the language is only set to grow. R has some intrinsic advantages that developers should be aware of.

Advantages of R Programming

The interest in this particular language only growing as gauged on the basis of indexes of popularity like PyPL, TIobe and Redmonk. R first made its appearance way back in the 1990s and was in the days of its inception an implementation as well as the adoption of the programming language for statistics, S. According to industry veterans R tops the list of programming languages in terms of popularity.

Experts like it due to the easy nature of its programming when viewed from level of computer science. With the passage of time R has gained much ground in terms of speed and serves as a programming language which binds together different sets of data, packages of software or other tools. It is considered to be the best way there is to create high quality of analysis of data that is reproducible. Experts like it for its power and flexibility that they need while dealing with Big Data. Some programmers even admit that most R programs run by them are no more than a collection of programming scripts that are separately organized as projects.

R also boasts of highly strong ecosystem package along with its inherent charting benefits. That is indeed one of its strong points. Programmers mince no words when it comes to lauding the vast ecosystem package that comes with it. If there is an existence of a statistical technique chances are that there is already an R package available for it.

Statisticians too Love R

Statisticians get a lot in terms of built in functionality when dealing with R. It is extensible and offers a rich set of functions that allows developers to build their own set of tools as well as methods for analysis of data. With its evolution R has even managed to attract the attention of those with a background in even humanities and biosciences.

Graphical Abilities

R veterans consider the power of its ability to churn out graphics to be simply unmatched. The ggplot2 and the dplyr packages deserve special mention for plotting and manipulation of data easier, respective.

To Sum Up

As a conclusion we may simply recall the statement by an R programmer stating that R had simple improved the quality of his life.

The Rise of the AI in Big Data

Posted on October 19, 2015February 9, 2019 by Dexlab

The researchers working at the MIT “Computer Science and Artificial Intelligence Laboratory” or abbreviated simply as CSAIL are all set to make human intuition out of the analysis of big data equation by enabling computers to choose from the set of features that are put into use in order to identify patterns in the data that may be considered to be predictive. This is dubbed as the “Data Science Machine” and as things have progressed so far the software prototype has managed to beat 615 of 908 competing teams vying for the same ability across no less than three competitions of data science.

Big Data may be considered as a complex and huge ecosystem that combines innovative processes from fields as diverse as storage, data analysis, curation, networking as well as search in addition to other functions and processes. As things stand much of analysis of big data is already algorithmic and automated but at the end of the day it is business users and data scientists who are needed in order to determine the particular dataset and analysis features which are required for visualization in the end and take action on the communicated data.

To put it simply at the end of the whole process humans are needed in order to make choices about data point combinations to chart out the relevant information.

The Data Science Machine is intended to naturally complement human intelligence and to make the most of the Big Data that is available for us waiting to be used.

The analysis of Big Data and Engineering of Features

As mentioned earlier actionable information lies at the hands of the big data scientist who is writing the code for analysis. It is this code that guides the analysis of the big data engine. In essence the advancement made by the MIT researchers is that not only does it serve to provide answers to questions regarding the data but also suggests additional questions accordingly.

This may be put into varied uses like to estimate the capacity of wind farms to generate power or making predictions about students who are likely to drop out of online courses.

5 Hottest Online Applications Inspired by Artificial Intelligence – @Dexlabanalytics.

The ultimate destination for all your data-related queries and assistance is DexLab Analytics. Being a premier Data Science training institute Gurgaon, DexLab Analytics takes pride in offering excellent data analytics courses for aspiring candidates.

Interested in a career in Data Analyst?
To learn more about Data Analyst with Advanced excel course – Enrol Now.
To learn more about Data Analyst with R Course – Enrol Now.
To learn more about Big Data Course – Enrol Now.
To learn more about Machine Learning Using Python and Spark – Enrol Now.
To learn more about Data Analyst with SAS Course – Enrol Now.
To learn more about Data Analyst with Apache Spark Course – Enrol Now.
To learn more about Data Analyst with Market Risk Analytics and Modelling Course – Enrol Now.

Twelve Great Free R Programming E-books

Posted on October 18, 2015October 4, 2017 by Dexlab

To Big Data enthusiasts R is word or rather a letter that needs no introduction. R programming is a programming language that brings the complex world of statistics and datasets at your fingertips. It is mainly used for computing statistics and relevant graphics. The following twelve e-books are not only useful to bring you up to the task for R programming but best of all they are free.

Learning Statistics with R
Author: Daniel Navarro

If you are looking for a guide that will take you through the intricacies of developing software with R be it the basic types and structures of data to more complex topics like recursion, closures as well as anonymous functions. Knowledge of statistics, although helpful, is not an essential pre-requisite .

Continue reading “Twelve Great Free R Programming E-books”

The Possibilities of Big Data

Posted on October 15, 2015May 30, 2017 by Dexlab

It is no secret that Big Data has some wonderful applications that may change the way we interact with businesses, and even more how they interacts with us through other facets of this rapidly growing field. But, what can it do concretely? This blog post shares insights of this question.

Endless Possibilities of Big Data

It can tell you what may most probably happen

Continue reading “The Possibilities of Big Data”

Top 10 Best Hadoop EBooks That You Should Start Reading Now

Posted on October 12, 2015May 23, 2020 by Dexlab

Based on Java, Hadoop is a free open source framework for programming where dealings with huge amounts of processed data in a computing environment is said to be distributed. None other than the Apache Software Foundation is sponsoring it. If you are looking for information about Hadoop, you will like to get in-depth information about the framework and its associated functions. To get you up to the mark with the concepts, the eBooks listed below will prove to be of invaluable help.

MapReduce

If you are looking forward to get started with Hadoop, and maximize your knowledge about Hadoop clusters, this book is of right fit. The book is loaded with information on how t o effectively use the framework to scale apps of the tools provided by Hadoop. This ebook lets you get acquainted with the intricacies of Hadoop with instructions provided on a step-by-step basis and guides you from being a Hadoop newbie to efficiently run and tackle complex Hadoop apps across a large number of machine clusters.

Also read: Big Data Analytics and its Impact on Manufacturing Sector

Programming Pig

If you are looking for a reference from which you may learn more about Apache Pig, which happens to be the engine powering executions of parallel flows of data on the Hadoop framework which also is open source, the Programming Pig is meant for you. Not only does it serve the interests of new users but also provides advanced users coverage on the most important functions like the “Pig Latin” scripting language, the “Grunt” shell and the functions defined by users for extending Pig even further. After reading this book, analyzing terabytes of data is a far less tedious task.

Also read: What Sets Apart Data Science from Big Data and Data Analytics

Professional Hadoop Solutions

This book covers a gamut of topics such as that how to store data with Hbase and HDFS, processing the data with the help of MapReduce and data processing automation with Oozie. Not limiting to that the book further covers the security features of Hadoop, how it goes along with Amazon Web Services, the best related practices and how to automate in real time the Hadoop processes. It provides code examples in XML and Java and refers to them in-depth along with what has been added to the Hadoop ecosystem of late. The eBook positions itself as comprehensive resource with API coverage and exposition of the deeper intricacies, which allow developers and architects to better customize and leverage them.

Apache Sqoop cookbook

This guide allows the user to use Sqoop from Apache with emphasis on application of parameters that are enabled by the Command Line Interface when dealing with cases that are used commonly. The authors offer Oracle, MySQL as well as PostgreSQL examples of databases on GitHub that lend themselves to be easily adapted for Netezza, SQL Server, Teradata etc relational systems.

Also read: Why Getting a Big Data Certification Will Benefit Your Small Business

Hadoop MapReduce Cookbook

The preface of the book claims that the book enables readers to know how to process complex and large datasets. The book starts simple but still gives detailed knowledge about Hadoop. Further, the book claims to be a simple guide on getting things done in one place. It consists of 90 recipes that are offered simply and in a straightforward manner, coupled with systematic instructions and examples from the real world.

Also read: How to Code Colour Values Within SAS Enterprise Guide

Hadoop: The Definitive Guide, 2nd Ed

If you want to know how to maintain and build distributed systems that are both scalable and reliable within the framework of Hadoop then this book is for you. It is intended for – programmers who want to analyze datasets, irrespective of size; and – administrators, who seek to know the setting up and running of Hadoop Clusters, alike. New features like Sqoop, Hive as well as Avro are dealt with in the new second edition. Case studies are also included that may help you out with specific problems.

Also read: How to Use PUT and %PUT Statements in SAS: 6 Tips

MapReduce Design Pattern

If one is to go by the book’s preface, the book is a blend of familiarity and uniqueness. The book is dedicated to design patterns by which we refer to the general guides or templates for solving problems. It is however more open-ended in nature than a “cookbook” as problems are not specified. You have to delve more in the subject matter than mere copying and pasting, but a pattern will get you covered about 90% of the whole way regardless of the challenge at hand.

Also read: SAS Still Dominates the Market After Decades of its Inception

Hadoop Operations

This book is necessary for those who seek to maintain complex and large clusters of Hadoop. Map Reduce, HDFS, Hadoop Cluster Planning. Hadoop Installation as well as Configuration, Authorization and authentication, Identity, Maintenance of clusters and management of resources are all dealt in it.

Programming Hive

Knowledge on programming in Hive provides an SQL dialect in order to query data, which is stored in HDFS, which makes it an indispensable tool at the hands of Hadoop experts. It also works to integrate with other file systems, which may be associated with Hadoop. Examples of such file systems may be MapR-FS and the S3 from Amazon as well as Cassandra and HBase.

Hadoop Real World Solutions CookBook

The preface of this eBook illustrates its use. It lets developers get acquainted and become proficient at problem solving in the Hadoop space. The reader will also get acquainted with varied tools related to Hadoop and the best practices to be followed while implementing them. The tools included in this cookbook are inclusive of Pig, Hive, MapReduce, Giraph, Mahout, Accumulo, HDFS, Ganglia and Redis. This book intends to teach readers what they need to know to apply Hadoop knowledge to solve their own set of problems.

So, happy reading!

DexLab Analytics Presents #BigDataIngestion

Besides, feeding knowledge through eBooks, it is vital to be enrolled for an excellent Big data hadoop certification in Gurgaon. DexLab Analytics is here for you; it offers a gamut of high-end big data hadoop training in Delhi, courses that will surely hone your data skills.

Interested in a career in Data Analyst?
To learn more about Data Analyst with Advanced excel course – Enrol Now.
To learn more about Data Analyst with R Course – Enrol Now.
To learn more about Big Data Course – Enrol Now.
To learn more about Machine Learning Using Python and Spark – Enrol Now.
To learn more about Data Analyst with SAS Course – Enrol Now.
To learn more about Data Analyst with Apache Spark Course – Enrol Now.
To learn more about Data Analyst with Market Risk Analytics and Modelling Course – Enrol Now.

Author: Dexlab