Mullvad Proxy

Free proxies, while seemingly profitable and attractive, are actually not very effective. They cannot boast of security, speed, stability and acceptable duration of work. Qualitative and reliable proxies require a certain investment, but they can be obtained from companies that have a good reputation as proxy service providers. You can also find out about all the nuances of proxy selection with the help of special proxy databases.

Go to the settings (the icon in the form of three dots) and open the section "Settings". In the tab at the very bottom, click on "Advanced settings". Click on "Open proxy settings for computer" and in the window that appears, click on "Network settings". Find the line "Automatic detection of parameters", uncheck it, and then, in the section "Proxy", activate the option "Use a proxy server". Enter the proxy host and port in the appropriate fields, and then click "Apply".

If you want to check the proxy's regionality, use a tool such as the proxy checker. You can either download the program or use it online. To perform the check, which allows you to determine not only the country and city, but also a number of other important indicators, you need to enter your username and password in the appropriate fields.

When scraping paginated content, fetching the "next page" usually involves extracting the URL of the next page from the HTML of the current page. In PHP, you can use a library like Simple HTML DOM Parser to parse HTML and extract the URL for the next page.

Here's an example of how you might scrape the next page URL using PHP

Install Simple HTML DOM Parser:

You can download it from sourceforge and include it in your project, or use Composer:

composer require sunra/php-simple-html-dom-parser

Write a PHP script to scrape the next page URL:

find('a.next-page-link', 0);

    if ($nextPageLink) {
        // Extract the href attribute (URL) from the link
        $nextPageUrl = $nextPageLink->href;
        return $nextPageUrl;
    } else {
        return null; // No next page link found
    }
}

// Example usage
$currentUrl = 'https://example.com/page1'; // Replace with the URL of the current page
$nextPageUrl = scrapeNextPageUrl($currentUrl);

if ($nextPageUrl) {
    echo "Next Page URL: $nextPageUrl";
} else {
    echo "No Next Page URL found.";
}

Replace the $currentUrl variable with the URL of the current page.

Adjust the HTML element selector ('a.next-page-link') based on the structure of the website you are scraping.

Run the script:

Execute the PHP script to see the URL of the next page.

To run GUI autotests in GitLab CI\CD using Docker, Selenium, and PyTest, you can follow these steps:

1. Create a .gitlab-ci.yml file in the root directory of your project. This file will define the pipeline and the jobs for your CI\CD process.

2. Configure the pipeline to use the appropriate image for your tests. In this case, you can use a Python image with the required dependencies installed.

3. Define the before_script section to set up the environment for the tests, including installing the necessary packages and downloading the required drivers for Selenium.

4. Define the test job to run the PyTest tests using the Selenium WebDriver.

Here's an example of a .gitlab-ci.yml file:

stages:
  - test

variables:
  SELENIUM_CHROME_DRIVER: '102.0.5005.62'
  SELENIUM_FIREFOX_DRIVER: '0.26.0'

image: python:3.8

cache:
  paths:
    - .venv
    - requirements.txt

before_script:
  - apt-get update -qq
  - apt-get install -y --no-install-recommends \
      build-essential \
      wget \
      xvfb \
      xvfb-run
  - pip install --upgrade pip
  - pip install --quiet --upgrade pytest
  - pip install --quiet selenium
  - pip install --quiet webdriver-manager
  - wget https://github.com/SeleniumHQ/selenium/releases/download/v${SELENIUM_CHROME_DRIVER}/chromedriver_linux64.zip
  - unzip chromedriver_linux64.zip chromedriver
  - wget https://github.com/SeleniumHQ/selenium/releases/download/v${SELENIUM_FIREFOX_DRIVER}/geckodriver-v${SELENIUM_FIREFOX_DRIVER}
  - mv geckodriver-v${SELENIUM_FIREFOX_DRIVER} geckodriver

test:
  stage: test
  script:
    - pytest tests/
  tags:
    - selenium
  artifacts:
    reports:
      - html
  only:
    - master
    - merge_requests

This .gitlab-ci.yml file defines a single stage called test that runs the PyTest tests in the tests/ directory. The before_script section installs the necessary dependencies, downloads the Selenium WebDriver for Chrome and Firefox, and sets up the environment for running the tests.

The tags: - selenium line ensures that the job runs on a runner with the selenium tag, which should have the appropriate Selenium WebDriver installed. The artifacts: reports: - html line enables the generation of HTML reports for the test results.

The only: - master - merge_requests line specifies that the tests should be run on every commit to the master branch and on every merge request.

Once you've set up the .gitlab-ci.yml file, commit and push it to your repository. Then, create a new merge request or push to the master branch to trigger the CI\CD pipeline and run the GUI autotests using Docker, Selenium, and PyTest.