Category: Email Extractor

Introduction

In the previous blog, we covered the basics of extracting emails from text using PHP and MySQL. Now, we’ll take it a step further by learning how to scrape websites for email addresses. Web scraping is a powerful technique used to extract data from websites, and with the right approach, you can gather email addresses from web pages effectively.

In this blog, we will walk you through setting up a basic web scraper using PHP to extract emails from a given website and store them in a MySQL database.

1. What Is Web Scraping?

Web scraping involves using a program to automatically extract data from websites. It’s useful for tasks like gathering contact information, tracking prices, or collecting large datasets. Email extraction is one of the common use cases for web scraping.

2. Legal Considerations for Scraping

Before you begin, it’s important to understand the legal aspects of web scraping. Many websites have terms of service that restrict scraping. Be sure to:

• Check the Website’s Terms: Make sure you have permission to scrape.
• Respect Robots.txt: This file tells scrapers what is and isn’t allowed to be scraped on a website.
• Ethical Scraping: Avoid overwhelming a server with too many requests in a short time.

3. Tools Needed for Scraping with PHP

For scraping, you can use the following tools and libraries:

• cURL: A popular PHP library used for making HTTP requests.
• PHP DOMDocument: A library that allows you to parse HTML and XML documents.
• MySQL: To store extracted emails.

4. Setting Up Your Environment

Make sure your environment is set up with PHP, cURL, and MySQL. You can use WAMP or XAMPP as your local server.

1. Install WAMP/XAMPP.
2. Verify that cURL is enabled in your PHP configuration (php.ini).
3. Create a MySQL database email_extractor with the emails table as we did in the previous blog.

CREATE TABLE emails (
    id INT AUTO_INCREMENT PRIMARY KEY,
    email_address VARCHAR(255) NOT NULL,
    source VARCHAR(255)
);

5. Writing the Scraper Using cURL

cURL is a powerful PHP library that allows you to make HTTP requests to websites. Here’s how to use it to get the HTML of a webpage:

<?php
// URL of the website to scrape
$url = "https://example.com";

// Initialize cURL session
$ch = curl_init($url);

// Set cURL options
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

// Execute cURL and store the result
$html = curl_exec($ch);

// Close cURL session
curl_close($ch);

// Display the HTML content
echo $html;
?>

6. Parsing the HTML with PHP DOMDocument

Once you have the HTML content of the page, you need to extract the email addresses from it. PHP’s DOMDocument can be used to parse the HTML.

<?php
// Load HTML content into DOMDocument
$dom = new DOMDocument();
@$dom->loadHTML($html);

// Get all anchor tags
$links = $dom->getElementsByTagName('a');

foreach ($links as $link) {
    // Check if the link contains 'mailto:' (common for email links)
    $href = $link->getAttribute('href');
    if (strpos($href, 'mailto:') !== false) {
        $email = str_replace('mailto:', '', $href);
        echo "Found email: $email\n";
    }
}
?>

This script will find any mailto: links in the page and extract the email addresses.

7. Extracting Emails Using Regex

Not all email addresses are in mailto: links. Some are embedded in the text, so we can use regular expressions to capture them.

<?php
$pattern = '/[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}\b/i';
preg_match_all($pattern, $html, $matches);

foreach ($matches[0] as $email) {
    echo "Found email: $email\n";
}
?>

This will scan the entire HTML content for any email patterns and display them.

8. Storing Extracted Emails in MySQL

Once you’ve extracted the emails, you can store them in the MySQL database as shown in the previous blog. Here’s a quick recap of how to insert the emails into the emails table.

<?php
$servername = "localhost";
$username = "root";
$password = "";
$dbname = "email_extractor";

// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);

// Check connection
if ($conn->connect_error) {
    die("Connection failed: " . $conn->connect_error);
}

foreach ($matches[0] as $email) {
    $sql = "INSERT INTO emails (email_address, source) VALUES ('$email', '$url')";
    $conn->query($sql);
}

$conn->close();
?>

This will save all the found email addresses into the emails table with the source URL.

9. Handling Multiple Pages

Many websites have multiple pages of content. To scrape them all, you can loop through the pages by dynamically modifying the URL or following pagination links.

Example:

for ($i = 1; $i <= 5; $i++) {
    $url = "https://example.com/page=$i";
    // Fetch and process each page
}

Conclusion

In this blog, we covered the process of scraping websites for email extraction using PHP. We learned how to:

• Use cURL to fetch webpage HTML.
• Parse HTML using PHP DOMDocument.
• Extract emails using mailto: links and regex.
• Store the results in a MySQL database.

Introduction

Email extraction is an essential technique in data collection, allowing businesses to gather email addresses for various purposes like marketing, lead generation, and customer outreach. In this blog, we will introduce the basics of extracting emails using PHP and MySQL, and walk through a simple example to get you started.

1. Importance of Email Extraction

Email extraction helps businesses build contact lists, target potential customers, and analyze communication patterns. It is especially useful for:

• Marketing: Gathering emails for email marketing campaigns.
• Lead Generation: Extracting emails from websites or documents to create leads.
• Customer Analysis: Storing customer emails for future reference or outreach.

2. Setting Up PHP and MySQL Environment

To start with email extraction, you need to set up your local PHP environment. You can use either WAMP for Windows or XAMPP for Mac. Here’s how to get started:

• Install WAMP/XAMPP.
• Ensure PHP and MySQL are configured correctly.
• Create a database in MySQL for storing extracted email addresses.

For this series, we will create a database email_extractor and a table emails to store the extracted data.

CREATE DATABASE email_extractor;

CREATE TABLE emails (
    id INT AUTO_INCREMENT PRIMARY KEY,
    email_address VARCHAR(255) NOT NULL,
    source VARCHAR(255)
);

3. Writing a Simple PHP Email Extractor

PHP makes it easy to extract emails from any text using regular expressions. A common function used for this is preg_match_all(), which can scan a block of text for email patterns.

4. Regular Expression for Email Extraction

The heart of email extraction lies in using the correct regular expression (regex) to match email patterns. A simple regex for matching emails looks like this:

$pattern = '/[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}\b/i';

This regex matches most standard email formats and is case-insensitive.

5. Example Code for Extracting Emails

Here is a basic PHP script to extract emails from a given text:

<?php
$text = "Contact [email protected] or [email protected]";
$pattern = '/[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}\b/i';
preg_match_all($pattern, $text, $matches);
print_r($matches[0]);
?>

This script will output the found emails in the text.

6. Storing Emails in MySQL

After extracting emails, it’s important to store them in a MySQL database for future use. You can connect PHP to your MySQL database and insert the extracted emails into a table.

7. Inserting Extracted Emails into the Database

Here’s how you can modify the above script to store the emails in a MySQL database:

<?php
$servername = "localhost";
$username = "root";
$password = "";
$dbname = "email_extractor";

// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);

// Check connection
if ($conn->connect_error) {
    die("Connection failed: " . $conn->connect_error);
}

foreach ($matches[0] as $email) {
    $sql = "INSERT INTO emails (email_address, source) VALUES ('$email', 'sample text')";
    $conn->query($sql);
}

$conn->close();
?>

This will save each extracted email into the emails table.

8. Understanding How the Code Works

• preg_match_all(): This PHP function searches the input text for matches to the email pattern defined by the regex.
• MySQL Insertion: After matching emails, the script inserts each one into the MySQL database along with its source.

9. Testing and Verifying Your Script

Once the script is running, you can test it by inputting text that contains email addresses. After running the script, you should check your MySQL database to ensure that the email addresses have been correctly inserted.

been correctly inserted.

Conclusion

In this first blog, we’ve introduced the basic concepts of email extraction using PHP and MySQL. We set up the environment, wrote a simple script to extract emails using regex, and stored the results in a MySQL database. This serves as a foundation for more advanced techniques we’ll explore in future blogs.