Tinyproxy Reverse Proxy

In the "System Settings" section, open the "Network" tab, and then, when you highlight the active connection, click "Advanced". Here, in the "Proxies" tab, tick only the HTTP proxy if you do not intend to use other types of proxies temporarily. Enter the address of your proxy server and its port in the designated fields and click "OK".

Text parsing is the collection of text information, which is then converted either to form a log file or to perform the task set by the developer.

To set a proxy on NOX, you can follow these steps:

1. Open NOX Player: Launch the NOX Player application on your computer.

2. Click on the "Menu" icon: Locate the Menu icon, which looks like three horizontal lines, in the top right corner of the NOX Player window. Click on it to open the menu.

3. Select "Settings": From the menu, click on the "Settings" option to open the settings panel.

4. Go to "Advanced Settings": In the settings panel, click on the "Advanced Settings" tab.

5. Scroll down to "Proxy Settings": In the Advanced Settings tab, scroll down to the "Proxy Settings" section.

6. Enable "Use Proxy": To enable the proxy, check the box next to "Use Proxy."

7. Enter the Proxy Address and Port: In the "Proxy Address" field, enter the IP address or hostname of your proxy server. In the "Proxy Port" field, enter the port number of your proxy server.

8. Configure additional settings (optional): If your proxy requires authentication, you can enter the username and password in the "Proxy Username" and "Proxy Password" fields.

9. Save your changes: Click the "Save" button to apply the changes and enable the proxy in NOX Player.

10. Restart NOX Player: After saving the changes, restart the NOX Player for the new proxy settings to take effect.

Please note that using a proxy may affect your internet connection speed and the performance of NOX Player.

To keep only unique external links while scraping with Scrapy, you can use a set to track the visited external links and filter out duplicates. Here's an example spider that demonstrates how to achieve this:

import scrapy
from urllib.parse import urlparse, urljoin

class UniqueLinksSpider(scrapy.Spider):
    name = 'unique_links'
    start_urls = ['http://example.com']  # Replace with the starting URL of your choice
    visited_external_links = set()

    def parse(self, response):
        # Extract all links from the current page
        all_links = response.css('a::attr(href)').extract()

        for link in all_links:
            full_url = urljoin(response.url, link)

            # Check if the link is external
            if urlparse(full_url).netloc != urlparse(response.url).netloc:
                # Check if it's a unique external link
                if full_url not in self.visited_external_links:
                    # Add the link to the set of visited external links
                    self.visited_external_links.add(full_url)

                    # Yield the link or process it further
                    yield {
                        'external_link': full_url
                    }

        # Follow links to other pages
        for next_page_url in response.css('a::attr(href)').extract():
            yield scrapy.Request(url=urljoin(response.url, next_page_url), callback=self.parse)

- visited_external_links is a class variable that keeps track of the unique external links across all instances of the spider.
- The parse method extracts all links from the current page.
- For each link, it checks if it is an external link by comparing the netloc (domain) of the current page and the link.
- If the link is external, it checks if it is unique by looking at the visited_external_links set.
- If the link is unique, it is added to the set, and the spider yields the link or processes it further.
- The spider then follows links to other pages, recursively calling the parse method.

Remember to replace the start_urls with the URL from which you want to start scraping.

Most often it is used to substitute your real IP address. An example of when this is needed: watching shows on Netflix that are only available to US users. A proxy can be used to make a user logging in from anywhere in the world will be identified by the IP address as a US user. Another option is to test your site through a local web server. A proxy in this case is used to intercept all the traffic in order to analyze it further for errors and failures.