Poxwalker Proxies

To configure a proxy on your MikroTik router, you need the Winbox software. The following steps must be done in the application: Open the "IP"-"WebProxy" sections. Check the box next to "Enabled". Enter the parameters of the proxy-server.

After that you have to specify the data of the proxy in the browser to be used. As an example, let's take Google Chrome. What you need to do:

Open the browser.
Click on the icon "?" in the upper right corner.
Go to "Settings".
Select the "Advanced" option.
Click the "System" tab.
Click on "Open proxy settings for your computer".
Click on "Network settings".
Activate the "Use proxy server" option.
In the tab that opens, specify the IP address of the proxy server. You should enter it in the field of the protocol to which the proxy server belongs.
Click the "OK" button to save your settings.

It is not possible to set up a proxy connection in the program itself. That is, you should configure it either through the regular settings of Windows, or by using third-party utilities to forward traffic (e.g., through ProxyCap).

Rotary proxies are proxies that cyclically change their real IP address. This is used to make it harder to track their location. The port usually changes as well. How this happens depends on the software used on the proxy server.

Shared proxies should be understood as IPs and port numbers available to everyone. That is, many users can use them simultaneously. The most unreliable and slowest option.

In Node.js, you can introduce delays in your scraping logic using the setTimeout function, which allows you to execute a function after a specified amount of time has passed. This is useful for implementing delays between consecutive requests to avoid overwhelming a server or to comply with rate-limiting policies.

Here's a simple example using the setTimeout function in a Node.js script:

const axios = require('axios'); // Assuming you use Axios for making HTTP requests

// Function to scrape data from a URL with a delay
async function scrapeWithDelay(url, delay) {
  try {
    // Make the HTTP request
    const response = await axios.get(url);

    // Process the response data (replace this with your scraping logic)
    console.log(`Scraped data from ${url}:`, response.data);

    // Introduce a delay before making the next request
    await sleep(delay);

    // Make the next request or perform additional scraping logic
    // ...

  } catch (error) {
    console.error(`Error scraping data from ${url}:`, error.message);
  }
}

// Function to introduce a delay using setTimeout
function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

// Example usage
const urlsToScrape = ['https://example.com/page1', 'https://example.com/page2', 'https://example.com/page3'];

// Loop through each URL and initiate scraping with a delay
const delayBetweenRequests = 2000; // Adjust the delay time in milliseconds (e.g., 2000 for 2 seconds)
for (const url of urlsToScrape) {
  scrapeWithDelay(url, delayBetweenRequests);
}

In this example:

• The scrapeWithDelay function performs the scraping logic for a given URL and introduces a delay before making the next request.
• The sleep function is a simple utility function that returns a promise that resolves after a specified number of milliseconds, effectively introducing a delay.
• The urlsToScrape array contains the URLs you want to scrape. Adjust the delay time (delayBetweenRequests) based on your scraping needs.

Please note that introducing delays is crucial when scraping websites to avoid being blocked or flagged for suspicious activity.