decor
 

Planet Big Data logo

Planet Big Data is an aggregator of blogs about big data, Hadoop, and related topics. We include posts by bloggers worldwide. Email us to have your blog included.

 

January 22, 2018


Curt Monash

The chaotic politics of privacy

Almost nobody pays attention to the real issues in privacy and surveillance. That’s gotten only slightly better over the decade that I’ve written about the subject. But the problems with...

...
InData Labs

3 Reasons to Adopt Data Strategy in 2018

It is true that data boom continues and the digital economy of 2018 is driven by data and the challenge of turning it into facts, insights, and trends. Data is the critical resource of our age, that is why today’s data strategy encompasses far more than how to store it. The most important functions of...

Запись 3 Reasons to Adopt Data Strategy in 2018 впервые появилась InData Labs.

 

January 21, 2018


Simplified Analytics

How is your marketing gearing up and making use of Digital?

For many businesses, digital has simply been the domain of marketing: web marketing, email marketing, search engine marketing, social media marketing, etc. It was lightly wrapped around existing...

...
 

January 19, 2018


Revolution Analytics

Because it's Friday: Principles and Values

Most companies publish mission and vision statements, and some also publish a detailed list of principles that underlie the company ethos. But what makes a good collection of principles, and does...

...

Forrester Blogs

The Recent NPS Debate – What You Should Really Know

Many clients have asked me about my thoughts on the recent blog post by Jared Spool who says Net Promoter Score (NPS) is a harmful metric. I then usually mention that he is making excellent points...

...

Revolution Analytics

Registration and talk proposals open Monday for useR!2018

Registration will open on Monday (January 22) for useR! 2018, the official R user conference to be held in Brisbane, Australia July 10-13. If you haven't been to a useR! conference before, it's a...

...
 

January 18, 2018


Revolution Analytics

A simple way to set up a SparklyR cluster on Azure

The SparklyR package from RStudio provides a high-level interface to Spark from R. This means you can create R objects that point to data frames stored in the Spark cluster and apply some familiar R...

...
Ronald van Loon

How Telcos can Fully Benefit from IoT and Eco-Systems

As the world is progressing towards an age of development in the Internet of Things (IoT) and other aspects related to it, the concept of the digital consumer is on the rise. Consumers of today want to experience the feasibility that is promised through this method. From experiencing customer support resources that fulfill their needs to having a seamless experience across platforms, consumers really want to experience the full taste of this development in technology. The high expectations of the digital consumer have meant that Telcos now have to understand consumer preferences and give a solution that is aligned with the needs of today’s consumer. Not only is this expected of Telcos operating currently, but they can also increase their revenue streams.

One way to fulfill the needs of consumers in a flawless manner is through the use of the IoT and ecosystems. Today, I will talk in depth about how Telcos can use these concepts to their benefit. I recently had the opportunity to join a few other notable names such as Dez BlanchfieldLillian Pierson and Ruven Cohen on a tour of the Ericsson Studio in Kista, Sweden. At the facility, we had a conversation with Elias Blomqvist, Strategic Product Manager at Ericsson. The conversation with one of the leading members of an organization playing the role of a protagonist in this wave towards the future influenced me into jotting down this article and using the insights garnered through the visit in crafting the content.

Today’s Clients Needs and Trends

The number one thing that Telcos need to understand and make room for is the needs and trends that are sought by clients today. Understanding these trends would lead to the development of a better Business Support System (BSS) that can eventually establish a Telco and establish their position as a prominent role player for the wave of technology that is set to dawn in the future.

Internet of Things

One of the most common trends present in the clients of today is the over-reliance of consumers on the growing concept of the IoT. The IoT can connect almost all devices, including houses and cars, to one server to make things even more comprehensive for the end user. The rising trend of the IoT gives a lot of opportunities for Telcos to establish themselves in this growing need for services. The implementation of the IoT has also meant that many new business systems are required by organizations. The traditional business system might just not suffice for this growing demand.

5G Monetization

With 5G technologies expected to phase out during the coming five years, Telcos have added responsibilities on their shoulders to monetize the system in a way that is both consumer-friendly and generates revenue for them. This would mean that the traditional BSS would have to be altered in a way that it accommodates to the growing advancements in connectivity. Researchers have predicted that there will be a high stream of money, over $250 Billion on an annual basis. With so much value in the cards, Telcos wouldn’t want to miss out on the opportunities here.

App Ecosystems

An application ecosystem is basically a unique set of processes and capabilities that provides the full lifecycle management need for an application. This spans throughout the period of creation of the app until the end of its life. An app ecosystem will basically involve all the processes in the application development, uploading it on the store, and compliance with network policies. The app user experience can vary today based on the user’s subscription options and the network itself. Both these factors play an important role in the digital economy, as the app user system is dependent upon them. Telco operators are in a great position to change the experience of app developers for the better. By providing services that cater to this need, operators can help developers in providing users with a seamless experience across multiple platforms. Research from Ovum Data has indicated that analytics is another key aspect looking to be explored by app developers. Through the use of big data analytics, operators are looking make data accessible for third parties.

What Can Telcos Do to Benefit from These Opportunities?

Telcos can surely do a lot to benefit from the opportunities that are present in this regard. Some of the key points that they can implement and try out include:

  • Telcos will have to focus on customer engagement, because it is really necessary for making a mark in this age of technological advancement.
  • Telcos will have to show that they are adapting to the requests from the market. The market requests coming from customers and clients alike should be given value by Telcos, and these requests should also be worked upon to bring about a change.
  • Working in an ecosystem with another partner is another necessary alteration that Telcos should adopt.
  • Delivering a seamless experience over multiple channels is something we discussed before and is really important for winning over customers in this age as a seamless experience is what most customers crave.
  • There is definitely a need for a new Business Support System (BSS). We will talk about this in more detail in the next section.

How Can Telcos Make These Changes?

With the changes already mentioned above, we can talk about what Telcos really have to do to meet these changes. There are certain key aspects that need to be taken into perspective. A business support system plays an important role in propelling a Telco forward in this age. The business support system needs to be significantly altered.

A business support system is basically a group of elements that are used by Telcos in gaining customer insight, generating new revenue methods and compiling real-time subscriptions. The top alteration that should be done in this regard is to direct the system towards managing customer experiences and feedback, rather than just subscription services. The transformation should really be directed to achieving a move from the demand-side of the vision to the supply-side of the vision. The new business support system should therefore be amended to meet the current trends in a more authentic manner. From point of sales to billing systems, a BSS manages everything, and hence innovation is deemed necessary here.

New Generation BSS

The new generation BSS should be focused upon providing value in the front end and should be all about customer engagement. This can be achieved through creating more intimate relations and an end to end solution. At the back end, the system should be enabled by cloud and flexible enough to meet changing consumer demands. Ericsson Revenue Manager was first to demonstrate charging for 5G networks, at MWC 2016

In conclusion, it can be mentioned that nobody can deny the opportunities that are present for Telcos. However, to achieve these benefits in a thorough manner, Telcos will have to implement a few alterations that can help them fully benefit from the IoT. To learn more download the Ericsson monetization report here

 

The data for this article has been garnered from a conversation with Elias Blomqvist, Strategic Product Manager at Ericsson. I captured this content on behalf of DevMode Strategies during an invite-only tour of the Ericsson Studio in Kista. Rest assured, the text and opinions are my own. I would like to thank them for giving me the privilege to visit their studio in Kista.

Ronald

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach.

He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation.

He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions.

Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining.

Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

More Posts - Website

Follow Me:
TwitterLinkedIn

Author information

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach. He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation. He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions. Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining. Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

The post How Telcos can Fully Benefit from IoT and Eco-Systems appeared first on Ronald van Loons.

 

January 17, 2018


Revolution Analytics

Microsoft R Open 3.4.3 now available

Microsoft R Open (MRO), Microsoft's enhanced distribution of open source R, has been upgraded to version 3.4.3 and is now available for download for Windows, Mac, and Linux. This update upgrades the...

...

Forrester Blogs

Modern CRM Drives Relationship And Revenue

CRM is more than two decades old. Companies initially used it to provide “inside-out” efficiencies – operational efficiencies for sales, marketing, and customer service organizations. Companies...

...

Forrester Blogs

Agency Layoffs Or Agency Calibration?

Layoffs Are A Reality Of The Agency Business Each January the industry weathers account losses, budget cuts or contract changes that result in  layoffs. Last week four agencies announced post-Holiday...

...
 

January 15, 2018

Ronald van Loon

Machine Learning Explained: Understanding Supervised, Unsupervised, and Reinforcement Learning

Once we start delving into the concepts behind Artificial Intelligence (AI) and Machine Learning (ML), we come across copious amounts of jargon related to this field of study. Understanding this jargon and how it can have an impact on the study related to ML goes a long way in comprehending the study that has been conducted by researchers and data scientists to get AI to the state it now is.

In this article, I will be providing you with a comprehensive definition of supervised, unsupervised and reinforcement learning in the broader field of Machine Learning. You must have encountered these terms while hovering over articles pertaining to the progress made in AI and the role played by ML in propelling this success forward. Understanding these concepts is a given fact, and should not be compromised at any cost. Here we discuss the concepts in detail, while making sure that the time you spend understanding these concepts pays off and that you are constantly aware of what is happening during this progress towards an Artificially Intelligent society.

Supervised, unsupervised and reinforcement Machine Learning basically are a description of ways in which you can let machines or algorithms loose on a data set. The machines would also be expected to learn something useful out of the process. Supervised, unsupervised and reinforcement learning lead the way into the future of machines that is expected to be bright, and will over time assist humans in doing everyday things.

Supervised Learning

Before we delve into the technical details regarding supervised learning, it is imperative to give a brief and simplistic overview that can be understood by all readers, regardless of their experience in this growing field.

With supervised learning, you feed the output of your algorithm into the system. This means that in supervised learning, the machine already knows the output of the algorithm before it starts working on it or learning it. A basic example of this concept would be a student learning a course from an instructor. The student knows what he/she is learning from the course.

With the output of the algorithm known, all that a system needs to do is to work out the steps or process needed to reach from the input to the output. The algorithm is being taught through a training data set that guides the machine. If the process goes haywire and the algorithms come up with results completely different than what should be expected, then the training data does its part to guide the algorithm back towards the right path.

Supervised Machine Learning currently makes up most of the ML that is being used by systems across the world. The input variable (x) is used to connect with the output variable (y) through the use of an algorithm. All of the input, the output, the algorithm, and the scenario are being provided by humans. We can understand supervised learning in an even better way by looking at it through two types of problems.

Classification: Classification problems categorize all the variables that form the output. Examples of these categories formed through classification would include demographic data such as marital status, sex, or age. The most common model used for this type of service status is the support vector machine. The support vector machines set forth to define the linear decision boundaries.

Regression: Problems that can be classified as regression problems include types where the output variables are set as a real number. The format for this problem often follows a linear format.

Unsupervised Learning

Since we now know the basic details pertaining to supervised learning, it would be pertinent to hop on towards unsupervised learning. The concept of unsupervised learning is not as widespread and frequently used as supervised learning. In fact, the concept has been put to use in only a limited amount of applications as of yet.

Despite the fact that unsupervised learning has not been implemented on a wider scale yet, this methodology forms the future behind Machine Learning and its possibilities. We always talk about ML bringing forth unlimited opportunities in the future, but fail to grasp the detail behind the statements made. Whenever people talk about computers and machines developing the ability to “teach themselves” in a seamless manner, rather than us humans having to do the honor, they are in a way alluding to the processes involved in unsupervised learning.

During the process of unsupervised learning, the system does not have concrete data sets, and the outcomes to most of the problems are largely unknown. In simple terminology, the AI system and the ML objective is blinded when it goes into the operation. The system has its faultless and immense logical operations to guide it along the way, but the lack of proper input and output algorithms makes the process even more challenging. Incredible as the whole process may sound, unsupervised learning has the ability to interpret and find solutions to a limitless amount of data, through the input data and the binary logic mechanism present in all computer systems. The system has no reference data at all.

Since we expect readers to have a basic imagery of unsupervised learning by now, it would be pertinent to make the understanding even simpler through the use of an example. Just consider that we have a digital image that has a variety of colored geometric shapes on it. These geometric shapes needed to be matched into groups according to color and other classification features. For a system that follows supervised learning, this whole process is a bit too simple. The procedure is extremely straightforward, as you just have to teach the computer all the details pertaining to the figures. You can let the system know that all shapes with four sides are known as squares, and others with eight sides are known as octagons, etc. We can also teach the system to interpret the colors and see how the light being given out is classified.

However, in unsupervised learning, the whole process becomes a little trickier. The algorithm for an unsupervised learning system has the same input data as the one for its supervised counterpart (in our case, digital images showing shapes in different colors).

Once it has the input data, the system learns all it can from the information at hand. In fact, the system works by itself to recognize the problem of classification and also the difference in shapes and colors. With information related to the problem at hand, the unsupervised learning system will then recognize all similar objects, and group them together. The labels that it will give to these objects will be designed by the machine itself. Technically, there are bound to be wrong answers, since there is a certain degree of probability. However, just like how we humans work, the strength of machine learning lies in its ability to recognize mistakes, learn from them, and to eventually make better estimations next time around.

Reinforcement Learning

Reinforcement Learning is another part of Machine Learning that is gaining a lot of prestige in how it helps the machine learn from its progress. Readers who have studied psychology in college would be able to relate to this concept on a better level.

Reinforcement Learning spurs off from the concept of Unsupervised Learning, and gives a high sphere of control to software agents and machines to determine what the ideal behavior within a context can be. This link is formed to maximize the performance of the machine in a way that helps it to grow. Simple feedback that informs the machine about its progress is required here to help the machine learn its behavior.

Reinforcement Learning is not simple, and is tackled by a plethora of different algorithms. As a matter of fact, in Reinforcement Learning an agent decides the best action based on the current state of the results.

The growth in Reinforcement Learning has led to the production of a wide variety of algorithms that help machines learn the outcome of what they are doing. Since we have a basic understanding of Reinforcement Learning by now, we can get a better grasp by forming a comparative analysis between Reinforcement Learning and the concepts of Supervised and Unsupervised Learning that we have studied in detail before.

  1. Supervised vs Reinforcement Learning: In Supervised Learning we have an external supervisor who has sufficient knowledge of the environment and also shares the learning with a supervisor to form a better understanding and complete the task, but since we have problems where the agent can perform so many different kind of subtasks by itself to achieve the overall objective, the presence of a supervisor is unnecessary and impractical. We can take up the example of a chess game, where the player can play tens of thousands of moves to achieve the ultimate objective. Creating a knowledge base for this purpose can be a really complicated task. Thus, it is imperative that in such tasks, the computer learn how to manage affairs by itself. It is hence more feasible and pertinent for the machine to learn from its own experience. Once the machine has started learning from its own experience, it can then gain knowledge from these experiences to implement in the future moves. This is probably the biggest and most imperative difference between the concepts of reinforcement and supervised learning. In both these learning types, there is a certain type of mapping between the output and input. But in the concept of Reinforcement Learning, there is an exemplary reward function, unlike Supervised Learning, that lets the system know about its progress down the right path.
  2. Reinforcement vs. Unsupervised Learning: Reinforcement Learning basically has a mapping structure that guides the machine from input to output. However, Unsupervised Learning has no such features present in it. In Unsupervised Learning, the machine focuses on the underlying task of locating the patterns rather than the mapping for progressing towards the end goal. For example, if the task for the machine is to suggest a good news update to a user, a Reinforcement Learning algorithm will look to get regular feedback from the user in question, and would then through the feedback build a reputable knowledge graph of all news related articles that the person may like. On the contrary, an Unsupervised Learning algorithm will try looking at many other articles that the person has read, similar to this one, and suggest something that matches the user’s preferences.

The realms in Machine Learning are endless. You can pay a visit to my YouTube channel to get to know more about the world of AI and how the future will be dictated by the use of data in machines.

Ronald

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach.

He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation.

He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions.

Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining.

Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

More Posts - Website

Follow Me:
TwitterLinkedIn

Author information

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach. He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation. He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions. Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining. Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

The post Machine Learning Explained: Understanding Supervised, Unsupervised, and Reinforcement Learning appeared first on Ronald van Loons.

 

January 14, 2018


Simplified Analytics

Role of Mobile in Digital Transformation

Our world has become increasingly digital.  The mobile-first strategy is no longer valid as we live in the mobile-only world. Today number of people having mobile phones on the planet is much...

...
 

January 12, 2018


Revolution Analytics

Because it's Friday: Kite Ballet

With a tip 'o the hat to Buck, enjoy the acrobatics of these kites from a performance in Oregon in 2012, set to Bohemian Rhapsody. Even after watching it a few times I still don't get how the lines...

...
Big Data University

Deploy Watson Conversation Chatbots to WordPress

If you’re reading this then you’ve most likely heard all the buzz around chatbots. In fact, you may have come up with a few scenarios where it would be really helpful for you to use one.

Most people consider chatbots to be in the realm of what only programmers can create, out of reach of business users who would otherwise have a need for them.

Thankfully, IBM provides the Watson Conversation service on their IBM Cloud platform which, combined with our WordPress plugin, solves that.

The plugin provides you with an easy way to deploy chatbots you create with IBM Watson Conversation to WordPress sites. In fact, you may have noticed a floating chatbot icon at the bottom of this page. Click on it to see the plugin in action.

What is Watson Conversation?

Watson Conversation is IBM’s chatbot service. Its intuitive interface allows chatbot creators to build their chatbot and have it ready to deploy in short time. You can sign up for a free IBM Cloud Lite account to get started.

Building your chatbot won’t be covered in this article but we have a great Chatbot course that guides you through this process and doesn’t require any coding expertise.

 

 

How do I add a chatbot to my website?

This is where the Watson Conversation WordPress plugin saves you time and money. If you have a website built using WordPress, deploying your chatbot to your website takes about 5 minutes and no code at all (as opposed to having to build your own application just to deploy a chatbot on the web.)

You can install it like any other WordPress plugin from your Admin page, that is, the first page you see after signing in.

 

 

Just search for Watson Conversation in the “Add New” section of the Plugins page and click “Install Now”.

Now you can find a page for “Watson” in your Settings. This is where you’ll find all the settings and customization to do with the plugin. When you first open it, you’ll see several tabs along the top.

For now, the only one you have to worry about is “Main Setup”.

 

 

You can find the credentials for the three required fields on the Deploy page of your Watson Conversation workspace.

 

 

Now just click save changes and you’re done. Browse your website and see your chatbot in action!

If you’re not quite satisfied with the appearance, you can customize this in the “Appearance” tab of the settings page.

You can also choose which pages to display the chat box on from the “Behaviour” tab. However, that’s not all you can do.

If you want to make the options clear to the user, you can create predefined responses to the chatbot messages for the users to select. The VOIP feature can connect users to your phone line over the internet from directly within the plugin.

In this brief article, we focused on how to deploy Watson Conversation chatbots to WordPress. Stay tuned for future articles on how to customize and use these exciting advanced features!

The post Deploy Watson Conversation Chatbots to WordPress appeared first on Cognitive Class.


Revolution Analytics

Services and tools for building intelligent R applications in the cloud

by Le Zhang (Data Scientist, Microsoft) and Graham Williams (Director of Data Science, Microsoft) As an in-memory application, R is sometimes thought to be constrained in performance or scalability...

...

Revolution Analytics

How to implement neural networks in R

If you've ever wondered how neural networks work behind the scenes, check out this guide to implementing neural networks in scratch with R, by David Selby. You may be surprised how with just a little...

...
 

January 11, 2018


Forrester Blogs

How State Farm Turned a Marketing Tagline into an Expression of Its Corporate Values

Happy 2018! I hope everyone enjoyed their holidays and had a chance to relax. As we get back to the swing of our regular routines, I have to wonder — what becomes of the season of giving? The...

...
Ronald van Loon

Artificial Intelligence Delivering a Personalized Content Experience

Artifical Intelligence is driving our efforts toward delivering a personalized content experience. Experience is the biggest enterprise disruption in 60 years. Experience is not some academic or grandiose idea.

Your friend’s and family’s behaviors are shaped by being consumers, whether they are interacting with technology on their mobile devices, at a bank kiosk, or using a touchscreen in retail or their car.

Digital is everywhere. We can tangibly see it in our everyday lives. This is changing the way companies organize themselves departmentally, and how they architect themselves technologically.

Enterprises need to change the way they think about technology. But the biggest organizational change becomes how you break down departmental silos, and put the customer at the forefront of what you are trying to do. Customers are only concerned with a consistent story from your enterprise that is personalized with what they are trying to achieve. But with the amount of data skyrocketing within organizations, how do you make real personalized experiences for customers?

John Mellor, who runs the Strategy and Business Development and Alliances Group at Adobe, gives us all a practical example in his everyday life…

IoT is quickly becoming a key technology in giving truly personalized experiences for customers. John travels often on a specific airline, who sends John alerts to his phone, such as when his luggage is being boarded. This is an IoT interaction because John’s luggage passed an IoT sensor that resulted in his phone being automatically alerted, which greatly improves his experience as an airline customer and traveler.

This is just one of millions of examples of how enterprises are improving the customer experience.

But what about from an organization’s perspective? How are technologies helping organizations overcome challenges when delivering a great and personalized customer experience?

Let’s take a look at Artificial Intelligence and Machine Learning.

It’s impossible for people to look at and understand the vast quantities of data being generated and determine trends or anomalies within that data.

But Artificial Intelligence and Machine Learning can watch data and spot trends or positive or negative anomalies. It facilitates in identifying offerings for consumer groups based on regions or demographics, for example. It helps enterprises operate efficiently and profitably because they make the customer experience better, which results in more loyal customers.

Technologies like AI and ML are essentially augmenting human tasks, making it easier to interact with customers. But the amount of data in an organization, or the algorithms put against that data, is no longer the greatest bottleneck to giving great personalized customer experiences.

Content now becomes the bottleneck to personalization. Finding enough content, breaking it down into subcomponents, and combining it with other content becomes the ultimate challenge for truly becoming personal with your audience.

Ronald

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach.

He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation.

He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions.

Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining.

Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

More Posts - Website

Follow Me:
TwitterLinkedIn

Author information

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach. He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation. He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions. Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining. Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

The post Artificial Intelligence Delivering a Personalized Content Experience appeared first on Ronald van Loons.


Revolution Analytics

R jumps to 8th position in TIOBE language rankings

The R language surged to 8th place in the 2017 TIOBE language rankings, up 8 places from a year before. Fellow data science language language Python also saw an increase in rankings, taking the 4th...

...
 

January 10, 2018

 

January 09, 2018


Revolution Analytics

In case you missed it: December 2017 roundup

In case you missed them, here are some articles from December of particular interest to R users. Hadley Wickham's Shiny app for making eggnog. Using R to analyze the vocal range of pop singers. A...

...
 

January 08, 2018


Revolution Analytics

Learn your way around the R ecosystem

One of the most powerful things about R is the ecosystem that has emerged around it. In addition to the R language itself and the many packages that extend it, you have a network of users,...

...

Forrester Blogs

Sales Digital Transformation: It’s Now Or Never!

Where is your company on this continuum of sales digital transformation? After writing about, speaking about, and advising clients on this topic for more than two and a half years, I believe we have...

...

Forrester Blogs

Forrester Forecasts 5.1% Growth In Global Tech Market In 2018 And 4.7% In 2019

Forrester projects that the global tech market for business and government purchases of technology goods, software, and services will grow by around 5% in 2018 and 2019 measured in constant...

...

Revolution Analytics

Divide and parallelize large data problems with Rcpp

by [Błażej Moska](https://www.linkedin.com/in/b%C5%82a%C5%BCej-moska-9a316113a/), computer science student and data science intern Got stuck with too large a dataset? R speed drives you mad? Divide,...

...
Ronald van Loon

Using Real Time Marketing and Machine Learning based Analytics to Drive Customer Value Management

The value of data-driven Customer Value Management or CVM cannot be underrated. Data and other algorithms/analytics that shape data are an imperative part of customer value management in a telecom company. With enhanced customer expectations, it is up to the ability of telecom companies to provide customers with a seamless experience and to also ensure that they help boost revenue in the process.

To understand this concept in a more functional manner, I recently interviewed the chief of CVM at Mahindra ComvivaAmit Sanyal. With so much on hand to discuss, I got to the crux of the matter straightaway and asked Amit about the pillars he considered to be important for a customer value management program being driven by analytics.

The prodigy responded to my questions by commenting that all methods of CVM being driven by the force of analytics should be dedicated towards these three pillars.

  1. Analytics themselves have an important part to play, which is why they form the first pillar in this regard. Understanding consumer behavior is not child’s play, so it is indeed profitable for a telecom company if the analytics are spot on in their methodology.
  2. The second pillar pertaining to efficiency in this regard points towards context in analytics. Analytics should attend to a derived need of consumers, and should be able to determine the channel of communication understanding the customer’s ‘sense’ is key here.
  3. The third pillar is that of real time communication. While secondary data collection is of immense importance itself, real-time primary communication cannot be understated based on the role it plays in understanding customers. Real-time means ‘as good as it gets in time’ – there is no merit in expecting customers to remember and act on propositions if not presented right when it is relevant, and more importantly ‘useful’.

Amit also outlined that one of the key challenges facing telecom companies globally is a drop in revenue. The drop in revenue is because of numerous reasons that are making growth a very difficult option to undertake for all protagonists involved in the market. While all telecom operators are looking out for newer options in the form of fresh customers, it is imperative to note here that fresh customers are rarely found. Most geographies have network connections than the people living in it or very close to that, so there is a real shortfall of new customers coming in for new connections. Other than the shortfall in garnering fresh customers, Amit also highlighted how the revenues from current customers were decreasing. These revenues have been decreasing steadily for a while now, due to the high amount of competition between the firms present. Most over the top or data content services are free. Margins have significantly dropped, since operators cannot risk selling at expensive rates considering how there are other operators selling at reduced rates.

The solution to this problem lies in reaching out to customers in a seamless manner. Since revenues in the market can only be increased through acquiring fresh customers or earning more revenue through current customers, reaching out to the customers and understanding their data is an inevitable outcome that needs to be followed.

Achieving Revenue

Since it has been mentioned above that there are limited fresh customers in the market, growth can only be achieved by bringing in customers from other operators. Simply put, you need to acquire customers from somewhere else to show your growth.

Besides bringing in new customers, you can also increase revenues from existing customers by understanding the economic concepts of elasticity and inelasticity. Operators need to know just what customers will be willing to spend their dollars on. Match the products and services they want with a price tag that gets customers to buy them.

Moreover, you can also increase the quality of your service. Subscribers tend to stay longer with an operator who offers quality. Not only will they stay longer, but they will also bring in new customers from other brands by telling them about the quality of your service.

To do all of this, you need to know just what the consumers are looking for. This is where the concept of machine learning and real time analytics come in. You should comprehend how your typical consumer behaves, and should also have a basic understanding of their preferences. You can use big data in the network systems, and implement methods such as predictive analytics and targeted communications to get the data that helps you understand them. By understanding their behavior and their preferences through real time data visualization, you can know just what will be perfect for your customers. Implementation of this method could open doors to data-driven customer value management and machine learning. Some of the stats pointing in favor of these changes are:

  1. Organizations that had real time data visualization enjoyed an increase of 26 per cent in their new identified pipeline accounts.
  2. Organizations that had implemented real time data visualization saw an increase of 15 per cent in the cash generated through operational activities.
  3. Engaging with customers is not just a beneficial tool for getting to know them but can eventually increase revenues and profitability. It has been found that if you engage with your customers, you will be able to generate 40 per cent more revenue per customer.
  4. Your marketing expenditure on personalization will not go to waste as it has been found that the tactic can increase your return on investment or ROI up to 5-6 times.
  5. A negative customer experience is nothing less than a cardinal sin. It takes more than 12 positive experiences to negate one negative impression that the customer must have developed through an experience.
  6. 70 per cent of all purchases are based on how the customer feels that they are being treated by the organization.
  7. 67 per cent of all customers leaving your organization could be stopped if you resolve their issue during the first engagement.
  8. Increasing customer retention by 2 percent is as beneficial as reducing costs by 10 per cent.

Data-driven marketing is key for enhancing the customer experience. Data driven marketing can help connect data points and link them together to create a more actionable context. Cases that highlight this are:

  • Customers don’t want to be told what to do. If a customer with a 4G phone is using 3G, they wouldn’t like the customer representative to tell them that they should switch to 4G. However, if the customer representative has sufficient data to see that they are using 3G since the last 3-4 months and also consumed most of the ‘data quota’ each month then he/she can recommend to them a 3G package with a higher band to increase satisfaction.
  • Using favorite and maximum recharge denominations data to get an indication of average revenue per user (ARPU). Telecom companies should study consumer data and know how much more they can spend and then offer them a feasible plan. Someone with an ARPU of $5 should not be given a $2,000 plan, instead they should be shown a $10 plan with hopes that it is within their extent of purchase.
  • All offers given to customers should be contextual. If a customer is spending time on international calls, then the offer given to them should be based on that and not driven by time.

The implementation of data-driven marketing calls for a mindset change in telecom operators. Operators need to understand what customers prefer, and then they should reach out to them on a personal level through data. “Everybody loves to talk about data science, it’s a cool thing – but only a few really move towards implementing it” said Amit before concluding the interview.

About the Author

Ronald van Loon is an Advisory Board Member and Big Data & Analytics course advisor for Simplilearn. He contributes his expertise towards the rapid growth of Simplilearn’s popular Big Data & Analytics category.

If you would like to read more from Ronald van Loon on the possibilities of Big Data and the Internet of Things (IoT), please click “Follow” and connect on LinkedIn and Twitter.

Ronald

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach.

He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation.

He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions.

Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining.

Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

More Posts - Website

Follow Me:
TwitterLinkedIn

Author information

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach. He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation. He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions. Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining. Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

The post Using Real Time Marketing and Machine Learning based Analytics to Drive Customer Value Management appeared first on Ronald van Loons.

InData Labs

Using Data Science to Grow Your Business: 3 Key Areas to Consider

Data science can have incredible benefits for your business but it’s important to understand that it’s a solution to a problem, not a way to find the problem. It means that if your company has a lot of data that you don’t quite know what to do with, you need to figure out what you...

Запись Using Data Science to Grow Your Business: 3 Key Areas to Consider впервые появилась InData Labs.

 

January 07, 2018


Simplified Analytics

Artificial Intelligence in Financial Lending

I remember the 90s when I wanted to get a home loan and it took me 3 months to complete the process from providing all the hard copies of my income, tax returns, identity proofs then bank...

...
 

January 06, 2018

Knoyd Blog

Behavioral Analysis of GitHub and StackOverflow Users

Which are the most actively used programming languages? Which website is used most often? What is the correlation in usage between the pages Github an StackOverflow?

In this blog post, we will take a look at the activity on websites that became a significant part of development across all areas in, as well as, outside of Data Science: GitHub and StackOverflow. It doesn't matter where developers are from or what their specific focus is, everyone uses these websites. Use cases are, for example, machine learning prototyping, data preparation, operationalization as well as software and web development. We have StackOverflow posts and GitHub commits from open repositories from 1.8.2017 to 15.9.2017. The StackOverflow data was downloaded from StackExchange and GitHub data was taken from Google Public Datasets using BigQuery. Firstly we will check activities on each website separately and afterward, we will look for correlations between the commits and StackOverflow activities.

 

GitHub

Now, let's look at the GitHub activity. 

Number of commits: 2317013
Number of programming languages: 318
git_commits

As expected we can see lower activity during the weekend. On the other hand, the peak is always on Wednesday. This means that developers are always the most active in the middle of the week and the number of commits is decreasing closer to the end of the week. The most often used languages in public repositories are python, shell, javascript, CSS, and HTML.

most_active_langs git_languages

We can see that commits of all popular languages follow the similar curves which reach the maximums in the middle of the week and bottoms during the weekend.

 

StackOverflow

The second source of data we had was from the StackOverflow website, specifically posts of members that were posted during the observed period. We didn't analyze answers or comments to the older posts.

Number of posts: 657199
Number of different tags: 26949

We can see that we have significantly less posts. On the other hand we have a lot more tags that make sense because tags are not limited to the programming languages.

stack_posts

We can see exactly the same pattern as on GitHub, where the number of posts falls approximately to 50% during the weekend with the peak on Wednesday. In this period, the most often used tags on StackOverflow were: javascript, python, java, android, and PHP.

most_active_tags stack_tags

Python and R

We will take a closer look at two programming languages closely related to Data Science: Python and R.

 

Python

We will take a look at the number of posts from StackOverflow with the tag "python" and number of commits from GitHub in repositories with language "python". We cannot compare the absolute numbers, therefore, we will normalize the values by subtracting the mean and dividing by the standard deviation. Furthermore, we will explore the percentage of "python" commits and posts respectively.
 

python python-percentage


We can see that after normalization, the python related activities follow almost the same curve with correlation coefficient 0.802. The only small difference is the deeper weekend bottoms of the StackOverflow curve. On the other hand, percentages are almost completely opposite, having a correlation coefficient  -0.718.

 


R
 

We will perform the same analysis on the language R.

R   r-percentage

We can see the same pattern in the number of commits and posts using R language and tag. The correlation coefficient is 0.807 which is almost the same as in Python. However, the percentages covered by the R language follow different pattern than Python, showing no dependency on each other, with correlation coefficient 0.167.

 

SUMMARY

We have taken a look at the languages used in GitHub and tags of posts on StackOverflow. We have identified that work of developers is always peaking on Wednesday, in the middle of the week. The used language has no effect on this, and the behaviour was always the same. Interestingly, percentages of commits and posts covered by the language Python vary and show exactly the opposite pattern, with a correlation coefficient -0.718.

 

January 05, 2018


Forrester Blogs

Thoughts on the Spectre of Zero Trust

The threat model has changed. Data breaches have traditionally required execution of some manner of code on a system to access data and a network connection to exfiltrate the data off the...

...

Forrester Blogs

Kicking Off The New Year With A MELTDOWN

What An Interesting Start To The Year I didn’t expect the year to kick off with it raining iguanas in Florida, a gas pumping crisis in Oregon, or the discovery and release of two massive CPU flaws...

...
 

January 04, 2018

 

January 03, 2018


Revolution Analytics

Make your R code run faster

There are lots of tricks you can use to make R code run faster: use more efficient data structures; vectorize your R code; offload complex data management tasks to databases. Emily Robinson shares...

...
 

January 02, 2018


Revolution Analytics

Do you have bad R habits? Here's how to identify and fix them.

RStudio's Jenny Bryan (whose recent interviews here and here you should definitely check out) has some excellent advice for improving your workflow for R: Use. Projects. — @JennyBryan at...

...

Forrester Blogs

Drive More Impact From Data And Analytics With Insights Storytelling

We said this a year ago, but alas, we have to say it again. There’s a big insights-to-actions gap out there.  Firms continue to invest in data, people, and technology, but in 2017, data and analytics...

...
 

January 01, 2018

Ronald van Loon

The Future is All about AI Devices That Can Actually Serve Us

The role of Artificial Intelligence (AI) devices in augmenting humans and in achieving tasks that were previously considered unachievable is just amazing. With the world progressing towards an age of unlimited innovations and unhindered progress, we can expect that AI will have a greater role in actually serving us for the better.

Since I have been associated with this wave of change towards AI-driven technologies and modules, I have literally been amazed at the ground we have covered during the last couple of years or so. As the technology behind AI gets revamped and updated on a regular basis, we can expect the wave of change to serve us in an even better way in the future.

A few cases of AI at work currently really do make us excited about the future of this technology. Some of the examples of this technology include:

  • We now have AI personal assistants to help us in tackling everyday tasks that were becoming a bit overwhelming in the past. These digital assistants can help streamline what you are doing, and come in handy to get your schedule on the right track. The potential for smart apps goes far beyond digital assistants. Many mobile applications are starting to make use of AI in a bid to improve performance and the satisfaction users eventually get.
  • A wide variety of linguists and other expert software developers dedicate time to build the services of a responsive personal assistant. These assistants can answer basic questions, while also tracking down information, sending messages, and launching services, among many other tasks.
  • Elsa is one AI app that is working to help users achieve a better English accent in a flawless manner. The app offers professional coaching and pronunciation practice that can help build your accent.
  • Google Allo follows on the trend of voice recognition to help you type text when you are on the move. Striking on the keyboard can often be exhaustive, so Google Allo is a good way to get the job done before exhausting yourself or diverting your attention. Reply to all those messages without moving a joint.

The Future

What we see in AI mechanisms and technology today is that they respond to what we say and do. However, it wouldn’t be unjustified to expect more innovations in this regard during the future. The future could see us witnessing smart devices that actually serve us, rather than responding to what we tell them to do.

Applications in the future could serve us by following open mobile and AI ecosystems. By serving us independently and by knowing us better, these devices would definitely be more intuitive and will provide a more convenient service.

Companies Driving the Future of AI

 

While we have been discussing possibilities in the future of AI, there are companies across the globe working tirelessly to achieve it. One such example can be of Baidu and Huawei. Both these organizations recently entered into an agreement that could lead the way into the future of Artificial Intelligence. The two companies aim to incorporate their offerings in a way that could benefit services ranging from AI Platforms, to Internet services and content. Both the organizations currently aim to build an open ecosystem through Baidu’s Brain and Huawei’s HiAI platform. The open ecosystem will eventually empower AI developers to explore bolder options by incorporating the services of both the companies. This will eventually open the door towards better AI offerings for consumers looking for a better smart experience.

About the Author

If you would like to read more from Ronald van Loon on the possibilities of Artificial Intelligence, Big Data, and the Internet of Things (IoT), please click “Follow” and connect on LinkedInTwitter and YouTube.

 

 

 

Ronald

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach.

He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation.

He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions.

Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining.

Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

More Posts - Website

Follow Me:
TwitterLinkedIn

Author information

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach. He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation. He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions. Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining. Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

The post The Future is All about AI Devices That Can Actually Serve Us appeared first on Ronald van Loons.

 

December 31, 2017


Simplified Analytics

How to develop the Vision for Digital Transformation?

Digital Transformation is now a number one priority for many businesses. Over the past two years, businesses have put increased focus on digitally transforming their brands from the inside out. It is...

...
 

December 28, 2017

Silicon Valley Data Science

Happy Holidays from SVDS

We look forward with anticipation to what 2018 will bring—and we wish you peace, prosperity, and happiness this season and in the year ahead.

 

The post Happy Holidays from SVDS appeared first on Silicon Valley Data Science.


BrightPlanet

Webinar Recap: How to Turn Web Content into Usable Data for Data Analytics

With technology and data, the possibilities are endless. However, when technology always changes, and the amount of content available online only increases, it becomes difficult to harvest the most relevant data. Many business owners become frustrated during the harvest data process with the complexity of ingesting unstructured content into their data models. At BrightPlanet, we […] The post Webinar Recap: How to Turn Web Content into Usable Data for Data Analytics appeared first on...

Read more »
 

December 26, 2017

InData Labs

4 Applications of Natural Language Processing that you Should Consider for your Business in 2018.

80% of the data generated by today’s businesses is unstructured. Most of it is generated from conversations with customer service representatives and on social media platforms. A lot of data about the companies can be found on review platforms such as Yelp or TripAdvisor and Q&A platforms such as Quora, as well as other Internet...

Запись 4 Applications of Natural Language Processing that you Should Consider for your Business in 2018. впервые появилась InData Labs.

 

December 25, 2017


Revolution Analytics

Merry Christmas and Happy New Year!

The Revolutions team is celebrating Christmas today, and we're taking a break with family and enjoying good food. And given the number of Eggnogs that are being prepared — thanks to Hadley Wickham's...

...

Simplified Analytics

How Digital is changing the workplace?

Remember the early 90s, when the mobile phones were not there. The only way to communicate to an out of office employee was landline phone or personal message via a colleague. Today there is no...

...
 

December 21, 2017


Forrester Blogs

Lost In The Dark On Marketing Measurement? We’ll Show You The Light.

Marketers have a new measurement mandate: to measure effectiveness across all channels and tactics by using advanced marketing measurement techniques. Yet, marketers remain in the dark; they are...

...

Revolution Analytics

A tour of the data.table package by creator Matt Dowle

The data.table package provides a high-performance interface for querying and manipulating data tables (a close cousin of data frames). In the video below (recorded at the H2OWorld conference),...

...

Forrester Blogs

Four Vendors Lead in The Forrester Wave: Enterprise Mobility Management, Q4 2017

Our recently released Forrester Wave: Enterprise Mobility Management, Q4 2017 uses 26 criteria to evaluate the top 13 mobility management solutions on the market today. Unlike in year’s past, this...

...

Forrester Blogs

Derisking the AI Worker

When you get over the fear of a robot taking over your job – because if you see our robots today they are still pretty dumb – your next big concern is how these new workers are going to...

...
Silicon Valley Data Science

Crossing the Development to Production Divide

Editor’s note: Welcome to Throwback Thursdays! Every third Thursday of the month, we feature a classic post from the earlier days of our company, gently updated as appropriate. We still find them helpful, and we think you will, too! You can find the original post here.

Many project teams have found themselves in the situation where it seems that they are “so close” to completing a product rollout, but can never quite seem to get there. It’s as if there exists an invisible canyon between development and production that only heroic feats can overcome (images of tightropes, rickety bridges, and just making a leap for it all come to mind). Once they finally cross they’ll have several bumps and bruises to commemorate the journey.

IT practitioners may find some comfort in the fact that they are not alone. A recent study by the Standish Group showed that 71% of projects are unsuccessful due to cost overruns, time overruns, or failure to deliver expected results; these are often symptoms of a lack of proper development practices (if you’re looking for a real life example, just ask the team behind the famously botched HealthCare.gov rollout). And, unfortunately, the advent of distributed computing as the backbone of big data solutions has only made this problem more challenging.

We’ve been there too. We know what it’s like to deal with complex production deployments that cover the gamut from infrastructure upgrades, to feature deployments, to data migrations, where each step threatens to derail the plan. In this post we’ll give an overview of obstacles we’ve faced (you may be able to relate) and talk about solutions to overcome these obstacles.

Causes of unsuccessful deployments

Often, we’ve found that the worst offenders are the seemingly innocent, small snags such as invalid configurations or outdated software versions that break code during runtime. In isolation, each snag isn’t a big deal; however, they can add up to a measurable overrun in total.

We have seen that these small snags are rooted in a couple of fundamental issues: big bang deployments where tasks cannot be worked incrementally and development environments that do not match production. While these issues are already being addressed by the DevOps community, they become especially troublesome and noticeable when working with distributed big data systems.

Often — especially when working with big data — development environments are intentionally small to save on cost. If your production environment requires 50 data nodes to house your 100 TB of data, you probably aren’t going to replicate the exact same setup in a development environment. The consequences can be many, for example:

    • Services are physically located in different places than they were in development
    • Data processing underperforms once scaled to production needs
    • Network latency and bandwidth constraints become noticeable at larger scales and as number of nodes increases
    • Polyglot architectures, while useful for ensuring the right tools are available for the right problems, increase the complexity of managing multiple technologies at once

And so on.

Thus, each time one of these issues crops up you can expect last minute fixes, configuration changes, and rework, causing your deployment to take much longer than anticipated.

Solution: architect for agility

Some may believe that more thorough planning is the key to overcoming unsuccessful production deployments. However, based on our experience, we can tell you that no amount of planning will overcome issues that are simply not apparent until you actually see how services perform in the real world.

Instead, one must architect for agility. One of the most important steps in this pursuit is to move production deployment up further in the process, while deploying in small increments, so that you get early and actionable feedback about how your services are going to perform. In keeping with our belief in the agile method, moving up deployment activities allows you to iterate much more quickly and frequently. It also alleviates some of the risk of individual tasks having a big impact on the timeline since you do not need to worry about releasing everything in “one big bang.”

Opportunities for further improvement

Essential to the notion of agile deployments is providing the capabilities that enable teams to more effectively deploy early and often.

At SVDS, we are investing in several ongoing initiatives that are designed to enable faster deployments and shorter feedback cycles:

      • “Push Button” infrastructure builds allow us to quickly spin up and tear down ‘production-like’ clusters at will, enabling us to test capabilities without fear of running up costs.
      • Monitoring frameworks allow us to refine data storage and performance requirements through the development processes, as well as proactively diagnose and take action on production issues in real time.
      • Automated test suites allow us to deploy new features and automatically regression test existing ones to ensure we do not impact stable code.

These capabilities allow our developers to immediately test features in production-like environments as they are developed and to obtain advance notice about areas where processes may have defects or may bottleneck on performance. If problems are discovered they can be addressed immediately before they become a real issue in production.

We are very excited about the development of these capabilities and look forward to sharing our progress as our work unfolds. We also invite the community to share its ideas on other best practices and initiatives in the comments below.

The post Crossing the Development to Production Divide appeared first on Silicon Valley Data Science.


Forrester Blogs

Agile Only ? No thanks ! Agile + DevOps, Please!

In 2017 — although I suppose I should say in 2018, as it’s almost the end of December — it is simply unacceptable for any IT organization to focus on an Agile-only or DevOps-only journey. They are...

...
Ronald van Loon

Digital Transformation Through Big Data, Analytics & Machine Learning

Can you imagine collecting data from 10,000 different data sources from over 65 billion records?

That’s exactly what RELX Group, a leading global information and analytics company, is doing to evolve with the Intelligent World. They have undergone a digital transformation to adapt to the ever changing digital landscape. They are achieving transformation through their control over Big Data and Analytics, and by using Machine Learning technology.

I spoke to Vijay Raghavan, the CTO for the LexisNexis Risk Solutions and Reed Business Information divisions at RELX Group, to find out how they help other organizations to make more informed, data driven decisions.

Whether RELX Group is working with a financial institution and needs to approve a mortgage loan. Relx Group is leveraging Big Data and Analytics and Machine Learning to help these very different industries address these very specific issues by creating or enhancing algorithms in order to garner insights from data.

Our world is being shaped by our technology. Data must be leveraged to make decisions so that businesses can evolve alongside the rapid pace of technology.

Ronald

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach.

He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation.

He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions.

Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining.

Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

More Posts - Website

Follow Me:
TwitterLinkedIn

Author information

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach. He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation. He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions. Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining. Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

The post Digital Transformation Through Big Data, Analytics & Machine Learning appeared first on Ronald van Loons.

 

December 20, 2017


BrightPlanet

The Washington Post Reporter Uses AMPLYFI to Research North Korea

BrightPlanet’s Data-as-a-Service (Daas) isn’t just for large-scale corporations or federal entities. Working with our partner, AI startup AMPLYFI, Pulitzer Prize winning journalist Joby Warrick put data harvesting technologies to work. Warrick sought to better understand the extent of North Korea’s bioweapons capability for a Washington Post article. AMPLYFI’s business intelligence product DataVoyant utilizes information harvested […] The post The Washington Post Reporter Uses AMPLYFI to...

Read more »

Forrester Blogs

Understand your continuous deployment maturity

At Forrester, we have developed an assessment to help organizations understand their continuous deployment maturity. The assessment should take 10 minutes or less to complete with the outcome...

...
 

December 19, 2017


Forrester Blogs

The Equifax Breach Will Haunt Us In Years To Come

Data breaches are now so common – and so large – that we measure them in percentage of worldwide internet users. Although Equifax doesn’t even make it into the top 5 at 4.08% of the approximately 3.5...

...

Revolution Analytics

ASA Police Data Challenge student visualization contest winners

The winners of the American Statistical Association Police Data Challenge have been announced. The ASA teamed up with the Police Data Initiative, which provides open data from local law enforcement...

...
Ronald van Loon

The Difference between Data Scientists, Data Engineers, Statisticians, and Software Engineers

Finding out the difference between data scientists, data engineers, software engineers, and statisticians can be confusing and complicated. While all of them are linked to data in a way, there is an underlying difference between the work they do and manage.

The growth of data and its usage across the industry is hidden from none. During the last decade in general, and the last couple of years in particular, we have seen a major distinction in the roles tasked with crafting and managing data.

Data Science is without a doubt a really growing field. Organizations and even countries from across the globe have experienced a drastic rise in their data collection endeavours. With numerous complications associated with collecting and managing data, this field is now host to a wide array of jobs and designations. We now have data scientists who are grouped into more specific tasks of data engineers, data statisticians, and software engineers. But other than the difference in their names, how many of us can comprehend the diversity in the work they do?

As I guessed, not many people can guess the job that these data experts are up to. Many of us eventually come to the conclusion that all of them do the same job and are grouped differently for the sake of it. There is nothing more mistaken then this myth and for this purpose, I have turned up as a myth buster today to put an end to the conflict in understanding the role of these jobs present in the data industry. While all of them help propel the movement towards authentic data creation by architecting the growth upwards, there is a major difference in how and why they come into the perspective.

Here I have outlined some of the major attributes of these four subcategories that come in the bigger picture of managing and looking over data. They say ignorance is bliss, but it is always better to know the real picture than to shy away from it.

Statistician

The statistician sits right at the forefront of the whole process and applies statistical theories to solve numerous practical problems pertaining to a plethora of industries. They have the leverage and the independence to determine the method deemed feasible for finding and collecting data.

Since statisticians are deployed to collect data through meaningful methods, they design surveys, questionnaires, experiments, etc., to collect data.

They analyze and interpret the analyses from the data and report all the conclusions that they find through their analyses to their superiors. Statisticians need to boast of analytic skills along with the ability to interpret data and narrate complex concepts in a simple, understandable manner.

Statisticians understand the numbers that are generated through research, and apply these numbers to real life issues.

Software Engineers

A software engineer sits at an important front of the data analytic process and is responsible for building systems and applications. Software engineers will be part of the process of developing and testing/reviewing systems and applications. They are responsible for creating the products that ultimately lead to the creation of the data. Software engineering is probably the oldest one of all these four roles and was an imperative part of society way before the data boom began.

Software engineers are responsible for developing frontend and backend systems that help collect and process data. These web/mobile applications lead to the development of the operation system through a flawless software design. The data that is generated through the apps created by software engineers is then passed on to data engineers and data scientists.

Data Engineer

A data engineer is someone who is dedicated towards developing, constructing, testing, and maintaining architectures, such as a large scale processing system or a database. The main difference between a data engineer and its often confused alternative data scientist is that a data scientist is someone who cleans, organizes, and looks over big data.

You might find the use of the verb “cleans” in the comparison above really exotic and inadvertent, but in fact, it has been placed with a purpose that helps reflect the difference between a data engineer and data scientist even more. In general, it can be mentioned that the efforts that both these experts put in are directed towards getting the data in an easy, usable format, but the technicalities and responsibilities that come in between are different for both of them.

Data engineers are responsible for dealing with raw data that is host to numerous machine, human, or instrument errors. The data might contain suspect records and may not even be validated. This data is not only unformatted, but also contains codes that work over specific systems.

This is where data engineers come in. Not only do they come up with methods and techniques to improve data efficiency, quality, and reliability, but they also have to implement these methods. To manage this complication, they will have to employ numerous tools and master a variety of languages. Data engineers actually ensure that the architecture that they work upon is feasible for data scientists to work with. Once they have gone through the initial process, the data engineers will then have to deliver or transfer the data over to the data scientist team.

In simple terminology data engineers ensure the flow of data in an uninterrupted way through servers. They are mainly responsible for the architecture needed by the data.

Data Scientists

We now know that data scientists will get data that has already been worked upon by data engineers. The data has been cleaned and manipulated and can be used by data scientists to feed analytic programs that prepare the data for its use in predictive modeling. To build these models, data scientists need to do extensive research and accumulate high volume data from external and internal sources to answer all business needs.

Once data scientists are done with the initial stage of analysis, they have to ensure that the work they do is automated, and that all insights are duly delivered to all key business stakeholders on a routine basis. It is indeed noticeable that the skill set needed for being a data scientist or a data engineer as a matter of fact is slightly similar. But the two are gradually becoming even more distinct within the industry. Data scientists need to know the intricate details related to stats, machine learning, and math to help build a flawless predictive model. Moreover, the data scientist also needs to know details pertaining to distributed computing. Through distributed computing, the data scientist will be able to access the data processed by the engineering team. The data scientist is also responsible for reporting to all business stakeholders, so a focus on visualization is necessary.

Data scientists use their analytical capabilities to find out meaningful extracts from the data that is being fed to the machine. They report the final results to all the key stakeholders.

The field of data is a growing one, and encompasses way more possibilities than what we had imagined before.

If you would like to read more from Ronald van Loon on the possibilities of Big Data, AI and the Internet of Things (IoT), please click “Follow” and connect on LinkedInTwitter and YouTube.

Ronald

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach.

He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation.

He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions.

Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining.

Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

More Posts - Website

Follow Me:
TwitterLinkedIn

Author information

Ronald helps data driven companies generating business value with best of breed solutions and a hands-on approach. He has been recognized as one of the top 10 global influencers by DataConomy for predictive analytics, and by Klout for Data Science, Big Data, Business Intelligence and Data Mining and is guest author on leading Big Data sites, is speaker/chairman/panel member on national and international webinars and events and runs a successful series of webinar on Big Data and on Digital Transformation. He has been active in the data (process) management domain for more than 18 years, has founded multiple companies and is now director at a Data Consultancy company, leader in Big Data & data process management solutions. Broad interest in big data, data science, predictive analytics, business intelligence, customer experience and data mining. Feel free to connect on Twitter or LinkedIn to stay up to date on success stories.

The post The Difference between Data Scientists, Data Engineers, Statisticians, and Software Engineers appeared first on Ronald van Loons.

 

December 18, 2017


Forrester Blogs

As CVS Goes Post-Digital, Walgreens Reverts to Pre-Digital

For a case study on how marketing must break from past practices, one needn’t look further than two recent announcements, one from CVS and the other from Walgreens. One company is pushing the...

...
decor