Skip to main content

Tutorial: Lead Contacts Scraper Agent

This agent is capable of scraping and extract email addresses, phone numbers, and social media profile links from a website domain in real-time. Can also retrieve data from major social platforms including Facebook, Instagram, TikTok, Twitter, LinkedIn, Youtube, Snapchat, Pinterest, and Github. In order to do this, you have to provide a valid URL to start the process.

RapidAPI Registration​

RapidAPI provides a variety of endpoints for various purposes. Among these, the website contacts scraper endpoint is utilized by this agent.

Follow the following steps to get your Rapid API key:

  1. Go to https://rapidapi.com/
  2. If you do not have an account, proceed to create one by following the registration process.
  3. After completing the sign up process, you will be assigned a default API key that you can access here.
alt text
  1. Copy your API key using the copy icon. You can also click the eye icon to see your API key.
alt text
  1. After your API key is copied, navigate to this link and go to the pricing tab.
alt text
  1. Select a subscription plan from the options available: Basic (free), Pro , or Ultra Plan. The Basic plan offers 100 API requests per month at no cost. For this example, let's opt for the Basic plan.
alt text
  1. Click the Subscribe button.
alt text
  1. Subscription created successfully.
alt text

Lead Scraper Agent Setup​

  1. Go to your SmythOS Dashboard and click on the Create Agent button.
alt text
  1. The template for the Lead Contacts Scraper is readily available. Simply select it from the drop-down menu.
alt text
  1. Click Create and allow the template to initialize and configure.
alt text

Adding Rapid API Key to the Vault​

Your API Key will be needed on your Website Contacts Scraper API Call component.

To know more about the Vault feature, please refer here.

  1. Navigate to the Vault section on your dashboard.
alt text
  1. Click the Add Key button to add your Rapid API Key.
alt text
  1. Input the Rapid API Key generated from your RapidAPI account.
alt text
  1. Select a distinct Key name that you can readily differentiate whenever it is required for use.
alt text
  1. Choose API Call for the Scope and click Done to save your API Key.
alt text
  1. Your Lead Scraper Rapid API Key has been successfully added to your Vault and is now ready for use.
alt text

Lead Scraper API Call Setup​

  1. Locate the Website Contacts Scraper API Call component that has a missing key component. Click on the component, then click the Edit button situated in the headers section.
alt text
  1. Highlight the text, then click on the key icon. Choose the Lead Scraper Rapid API Key stored in your Vault, and finally, click the Save button.
alt text
  1. Once saved, your API key is now added in the headers section. Additionally, don’t forget to click the checkmark icon to save your configuration.
alt text
  1. The red error mark on your Website Contacts Scraper API Call Component will disappear once your API key is successfully configured.
alt text
  1. You should now be able to use this agent to extract Emails, Phone Numbers, and Social URLs (Facebook, TikTok, Instagram, Twitter, LinkedIn and others) from a website in real-time.

Test the Agent​

Follow the steps outlined below to evaluate the Lead Contacts Scraper Agent.

  1. Open the ChatBot embodiment and click on the chat icon.
alt text
  1. Provide a valid URL of the website that you want to scrape data from.
alt text
  1. Here’s the result!

Here are the contact details and social media profiles I have found from the domain you provided:

Emails:

Phone Numbers:

  • 1661857200
  • 1661871600
  • 1594353335
  • 1594382400
  • 1594412149
  • 62022083111

Social Media Profiles:

  • Facebook
  • Twitter
  • LinkedIn
  • Pinterest
  • There are no available links for Instagram, Tiktok, Snapchat, Youtube, and Github.