Understanding the Importance of the Deep Web and Dark Web

During the course of a 2 day conference I recently attended, I had the opportunity to meet many security professionals from various organizations in the private sector and public sector.  In the hopes of understanding their perspective of the Deep Web and Dark Web, I posed several questions about the Deep Web and surprisingly, many people I spoke to had never heard of the Deep Web or the Dark Web while others conflated the two. This presented me the opportunity to share what each are and how they differ as well as the importance of the Deep Web in information security.

So what is the Deep Web? By definition, the Deep Web are web pages that are not indexed by search engines like Google, Yahoo or Bing. Taking the definition from http://whatis.techtarget.com/definition/deep-Web

“The deep Web is the part of the Internet that is inaccessible to conventional search engines, and consequently, to most users. According to researcher Marcus P. Zillman of DeepWebResearch.info, as of January 2006, the deep Web contained somewhere in the vicinity of 900 billion pages of information. In contrast, Google, the largest search engine, had indexed just 25 billion pages.”

Surface Web vs Deep Web

Let’s have a practical understanding on the difference between the Surface Web and the Deep Web. When you ‘google’ for Gmail, you will be presented with various results that eventually will direct you to the front page of Gmail.com. Now what you are seeing is actually the Surface Web. When you log in to Gmail.com with your credentials, you will be directed to another page where all your personal emails are stored. This Gmail inbox that you are now accessing is part of the Deep Web. So by imagining the millions of Gmail, Yahoo or Hotmail email subscribers, you can imagine how large the Deep Web is.

Deep Web vs Dark Web

Many quickly assumed that the Deep Web is a dark place where criminal activities and illicit services lurks hence many articles and reports posted by journalists and bloggers referring to the Deep Web and Dark Web synonymously. The Dark Web is part of the Deep Web but the Deep Web is NOT the Dark Web. Imagine this for a moment, a tomato is part of a salad but a tomato is not a salad by itself.  This “part of” not “same as” relationship is essential to understand.

The ‘Dark’ Net largest network, TOR

The Dark Web, sometimes called the Dark Net is accessible through special software and configurations.  As part of the Deep Web, the Dark Web is also accessible via standard search browsers. The largest private network that constitutes part of the Dark Web is accessed through software called The Onion Router (TOR) browser. The TOR browser, a project started by the United States Navy Research Labs connects a user to the TOR network which is essentially a microcosm of the same activities that exists in the surface net that we all use every day.  The TOR browser aims to provide a sense of anonymity when accessing the TOR network preventing monitoring of user activities throughout the network. With this sense of anonymity, cyber criminals have taken advantage of this technology and use it to conduct criminal activities such as providing illicit services, selling, and sharing 0 days cyber exploits, drugs, weapons, and other illicit activities. Since all of the illicit and legal activities occur with anonymity and essentially in the shadows, the nomenclature of the “Dark Web” was born. 

The Importance of Dark Web

The Dark Web is known to be a playground of illicit services and it has grown to be more than just a place to purchase drugs or trade 0 day exploits. It is also a place where stolen personal information and documents are leaked/dumped or auctioned.  Recent high profile cases such as the OPM (Office of Personnel Management) hack, Ashley Madison hack and just recently the Talk Talk hack are some of the cases where hackers are utilizing the Dark Web to sell or expose the data causing reputational damage to the affected organizations and great distress to the individuals that have their personal information exposed. There are multiple forums dedicated to the sale of these types of stolen personal information and those posting and making the sale shroud themselves in the anonymity that the Dark Web provides.

Are you already in the Deep Dark Web?

Throughout my research in identifying and gathering information related to several financial organizations and governments, I have managed to locate important and highly sensitive information. From the exposure of vulnerabilities, email addresses, credentials, personal information in private emails, databases, and even potential cyber threats all being leaked and exposed in the Deep Web. This information, in the wrong hands can lead to various implications such as phishing, ransom, threats, and even compromising the state of security of the affected organizations.

The majority of cyber threat intelligence data focus on preventing a breach or leak from happening, but I have found that even with companies and governments spending more and more on the defense of the network, major breaches are occurring at an even more alarming rate.  We must continue to try and protect one of the most valuable assets of the company, the information.  We must also realize that there will be times when our information makes its way onto the Dark Net where cyber criminals and black hat hackers can use the information to further compromise our companies and nations.  I believe we must continuously monitor the Dark Net, so we learn that our sensitive information is for sale or posted as early as possible.  This allows us to take proactive steps to begin remediation prior to a greater exploit or attack.

This blog can also be found on Linkedin: https://www.linkedin.com/pulse/understanding-importance-deep-web-dark-fadli-b-sidek