Tell me about the 'Deep Web'.

Do you use it?

What do you use it for?

What app is best to access it?

What should I know before I venture in?

Explanation of “Deep Web”.

Yep. First place I looked. :slight_smile:

I guess I’m more interested in personal experiences. Which may be better suited for IMHO.

How up-to-date is that article?

I’m just about completely ignorant of internet matters and have some questions. Lets say I want to create a website on WWII battleships and create pages of pics and text on it. IIRC one must register a website name like “WWIIBBS.org” with the appropriate agency and then find servers to hold my data and share it with folks who visit the page. Doesn’t this automatically make my website part of the surface web, showing up in google, bing, or yahoo searches? Or is it part of the DEEP WEB until it gets some links back to it from other pages?

Once you register the domain name, you’re part of the www (surface web) and can be crawled/googled.

You can make your site known to the major search engines manually. Google provides instructions on doing this here. Bing makes it even easier. (Yahoo.com shows Bing’s results.) If any page known to a search engine links to yours, the search engine’s ‘spider,’ or automated bot, will eventually find that link and follow it, making this process unnecessary. High traffic websites are indexed in this way multiple times a day, while more remote areas of the web are still indexed on a pretty regular schedule.

It seems likely that any commercial service you use to host your website would also send similar index information to major search engines periodically, though I don’t know for certain that this happens.

If registering my domain makes it part of the surface, how do I make it part of the deep web? Can I have a web page that is not part of a domain?

The deep web is stuff that is not crawled by the search engines. You can prevent search engines from crawling your site in a lot of ways.
1 Explicitly tell the engines not to crawl your site using the standard robots.txt file.
2 Navigate around your site via something other than links. Like using generated content based on user inputs. The user types something into a text box then the new web page is generated. Search engines start at the base page on your site and then start following links on the page to other pages on your site. If those links don’t exist then the engine has difficulty finding the other pages on your site.
3 Require a login to get to your site and don’t provide this to the search engines.

I’m learning as I go:

Someone’s been watching House of Cards…

You probably already do use the Deep Web, or at least some very specific portions of it. Web-based email is one example of the Deep Web: You can read your messages online, but nobody else (without your password) can.

I think that’s an example of “dark web”, not deep web.

A company that posts job listings on their web site is an example of the dark web because you can’t surface those job postings through google - only the site’s specific search engine.

Deep web, on the other hand, requires a specific browser, like TOR and finding links to places you might want to visit. Most of which seem pretty sketchy.

I could host a Web site on my home computer with no public domain name or host needed, just an IP address. You would need to traverse my ISP’s network to get to it so it’s not like it’s totally off the grid, but there’s no need for me to publicize it. I could give you the IP address and there you are.

The Master Himself has discussed this subject:

How can I access the deep, dark Web?

So is there anything that is NOT sketchy or illegal on the deep web?

I mean, I know there’s the nasty stuff, and the ‘lets all buy drugs with bitcoins’ stuff, and the super paranoia and spy stuff, but really, is that it?

It honestly just sounds like all the sketchy parts of a real town - what’s the allure? I don’t want to wander the bad parts of Ponce in Atlanta, because I might get knifed, and there’s nothing there I particularly need or want.

Why would I jump on TOR and head to a bad part of the internet where my computer/identity might get knifed?

Buying drugs with bitcoins and stuff. There is no allure, unless you want what is there.

Why would you go anywhere unless it has something you need or want? Weird question.

Already answered. Most of the Deep Web is boring stuff like email or payroll files or the like. Now, the Dark Web is considerably sketchier, but that’s really just by definition: The non-sketchy bits are non-dark.

The deep web is just websites that are not supposed to be publicly available.

Sometimes when doing academic research, I’ll play “guess the URL” to find missing reports. Like if an organization lists reports for 2009-2014 using something like http//:565example.report/2014, I may be able to find the 2008 report even though it’s not linked anywhere. It’s not anything shady- I’m just looking for public reports that are no longer actively published.

That was indeed the question - was there, in fact, anything OTHER than the sketchy/illegal stuff.

If not, I’m perfectly happy out here on the light-side.