banner
soapffz

soapffz

github
steam
bilibili
douban

Import Shodan exported JSON data into MongoDB database.

Background#

End of the year, just messing around.

Last time, I was lucky enough to get a lifetime membership to Shodan for $1. I can download data 20 times a month, with a limit of 10,000 records each time.

I'm bored, so I'm saving some data from Shodan that can be searched for PHPMyAdmin.

Disclaimer: The data used in this article is from legitimate sources and is only for testing purposes. I have deleted it within 24 hours.

Downloading Data from Shodan#

Shodan is a well-known network space security engine that mainly searches for various databases and websites publicly available on the internet.

It can be used in conjunction with Metasploit (msf), but personally, I find the quick jump navigation on the left side of the web page more convenient.

image

You can see that there are about 11,000 PHPMyAdmin servers in China that can be searched on the internet. Luckily, with my Shodan membership, I can download 10,000 records.

So, I downloaded 10,000 records.

When downloading, I chose the JSON format as XML is no longer recommended.

Let's take a look at the downloaded data. It appears to be arranged line by line.

image

I tried using Navicat's import function to directly import the formatted JSON data, but it failed. So, I directly imported it into the MongoDB database.

Importing into MongoDB Database#

I also tried importing the MongoDB database using Navicat, but the JSON data was not standardized.

The installation of MongoDB has been covered in a previous article: "Installing MongoDB on Win10".

After installation, we enter the default path of MongoDB:

C:\Program Files\MongoDB\Server\4.2\bin

Open the command prompt (cmd) at this location and enter:

mongo.exe

This opens the MongoDB command line interface. Let's create a database:

use shodan

Exit the interface and use mongoimport.exe to import the JSON data:

mongoimport.exe --db shodan_data --collection phpmyadmin_data --file E:\share\shodan-export.json\shodan_data.json

db is the database we created (you can also specify it without creating one), collection is the collection name, similar to a table in MySQL.

file specifies the downloaded JSON file from Shodan:

image

Importing takes only 2-3 seconds. Let's take a look using Navicat:

image

Well, the data has been imported successfully. Now, we can start batch processing... Wait, what was I saying?

PS: Navicat is an excellent tool for displaying databases, but it really lacks in data importing.

That's all for this article.

Loading...
Ownership of this post data is guaranteed by blockchain and smart contracts to the creator alone.