“Python Learning: Automate Boring Stuff with Python | Chapter 11 : My Solution to Link Verification (with a twist)

Instead of downloading the pages, let’s just write the links out

4:40:41 PM

In the last task we downloaded images directly to our machine into a folder in the current working directory. Since I am well acquainted with downloading files using requests, I decided to just write out the links into a text file instead of downloading the entire file.

The code is down below:

# Linked Verification -> Write all linked or 404 pages in a given page
# USAGE python linkver.py 

import logging
import sys
import requests
import bs4
import os

logging.basicConfig(level=logging.DEBUG,
                    format="%(asctime)s - %(levelname)s - %(message)s")

if len(sys.argv) == 2:
    # TODO: get url
    url = sys.argv[1]
    # TODO: get page
    try:
        res = requests.get(url)
        res.raise_for_status()
        # TODO: get all links
        page_soup = bs4.BeautifulSoup(res.text, "html.parser")
        page_links = page_soup.select("a")
        # TODO: create and write links into respective text files
        good_file = open("good_links.txt", "w")
        bad_file = open("bad_links.txt", "w")
        good_file.close()
        bad_file.close()
        # TODO: create and write links into respective text files
        good_file = open("good_links.txt", "a")
        bad_file = open("bad_links.txt", "a")
        # TODO: make requests to all links
        for link in page_links:
            link = link.get("href")
            if not link.startswith("http"):
                link = url + link
            try:
                link_res = requests.get(link)
                # TODO: get status code
                status = link_res.status_code
                # TODO: update 404 and good pages list
                if int(status) == 200:
                    good_file.write(link + "\n")
                    logging.info("Good links written")
                else:
                    bad_file.write(link + "\n")
                    logging.info("Bad links written")
            except Exception as err:
                logging.error("Page Link Error: " + str(err))

        good_file.close()
        bad_file.close()
    except Exception as err:
        logging.error("Url Error: " + str(err))
else:
    logging.error("USAGE python linkver.py ")

Result

So I ran :

python linkver.py https://www.lipsum.com/

and the logs:

2020-04-24 21:36:13,372 - DEBUG - Starting new HTTPS connection (1): www.lipsum.com
2020-04-24 21:36:14,156 - DEBUG - https://www.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:14,218 - DEBUG - Starting new HTTP connection (1): hy.lipsum.com
2020-04-24 21:36:14,567 - DEBUG - http://hy.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:14,571 - DEBUG - Starting new HTTPS connection (1): hy.lipsum.com
2020-04-24 21:36:15,064 - DEBUG - https://hy.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:15,074 - INFO - Good links written
2020-04-24 21:36:15,083 - DEBUG - Starting new HTTP connection (1): sq.lipsum.com
2020-04-24 21:36:15,433 - DEBUG - http://sq.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:15,437 - DEBUG - Starting new HTTPS connection (1): sq.lipsum.com
2020-04-24 21:36:15,912 - DEBUG - https://sq.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:15,921 - INFO - Good links written
2020-04-24 21:36:15,924 - DEBUG - Starting new HTTP connection (1): ar.lipsum.com
2020-04-24 21:36:16,273 - DEBUG - http://ar.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:16,281 - DEBUG - Starting new HTTPS connection (1): ar.lipsum.com
2020-04-24 21:36:16,775 - DEBUG - https://ar.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:16,785 - INFO - Good links written
2020-04-24 21:36:16,791 - DEBUG - Starting new HTTP connection (1): ar.lipsum.com
2020-04-24 21:36:17,014 - DEBUG - http://ar.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:17,018 - DEBUG - Starting new HTTPS connection (1): ar.lipsum.com
2020-04-24 21:36:17,499 - DEBUG - https://ar.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:17,507 - INFO - Good links written
2020-04-24 21:36:17,510 - DEBUG - Starting new HTTP connection (1): bg.lipsum.com
2020-04-24 21:36:17,874 - DEBUG - http://bg.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:17,878 - DEBUG - Starting new HTTPS connection (1): bg.lipsum.com
2020-04-24 21:36:18,349 - DEBUG - https://bg.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:18,365 - INFO - Good links written
2020-04-24 21:36:18,378 - DEBUG - Starting new HTTP connection (1): ca.lipsum.com
2020-04-24 21:36:18,732 - DEBUG - http://ca.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:18,736 - DEBUG - Starting new HTTPS connection (1): ca.lipsum.com
2020-04-24 21:36:19,209 - DEBUG - https://ca.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:19,219 - INFO - Good links written
2020-04-24 21:36:19,223 - DEBUG - Starting new HTTP connection (1): cn.lipsum.com
2020-04-24 21:36:19,571 - DEBUG - http://cn.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:19,575 - DEBUG - Starting new HTTPS connection (1): cn.lipsum.com
2020-04-24 21:36:20,054 - DEBUG - https://cn.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:20,062 - INFO - Good links written
2020-04-24 21:36:20,071 - DEBUG - Starting new HTTP connection (1): hr.lipsum.com
2020-04-24 21:36:20,422 - DEBUG - http://hr.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:20,427 - DEBUG - Starting new HTTPS connection (1): hr.lipsum.com
2020-04-24 21:36:20,910 - DEBUG - https://hr.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:20,919 - INFO - Good links written
2020-04-24 21:36:20,928 - DEBUG - Starting new HTTP connection (1): cs.lipsum.com
2020-04-24 21:36:21,273 - DEBUG - http://cs.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:21,278 - DEBUG - Starting new HTTPS connection (1): cs.lipsum.com
2020-04-24 21:36:21,758 - DEBUG - https://cs.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:21,768 - INFO - Good links written
2020-04-24 21:36:21,775 - DEBUG - Starting new HTTP connection (1): da.lipsum.com
2020-04-24 21:36:22,125 - DEBUG - http://da.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:22,131 - DEBUG - Starting new HTTPS connection (1): da.lipsum.com
2020-04-24 21:36:22,609 - DEBUG - https://da.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:22,618 - INFO - Good links written
2020-04-24 21:36:22,626 - DEBUG - Starting new HTTP connection (1): nl.lipsum.com
2020-04-24 21:36:22,977 - DEBUG - http://nl.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:22,981 - DEBUG - Starting new HTTPS connection (1): nl.lipsum.com
2020-04-24 21:36:23,464 - DEBUG - https://nl.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:23,474 - INFO - Good links written
2020-04-24 21:36:23,482 - DEBUG - Starting new HTTP connection (1): www.lipsum.com
2020-04-24 21:36:23,707 - DEBUG - http://www.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:23,713 - DEBUG - Starting new HTTPS connection (1): www.lipsum.com
2020-04-24 21:36:24,201 - DEBUG - https://www.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:24,210 - INFO - Good links written
2020-04-24 21:36:24,218 - DEBUG - Starting new HTTP connection (1): et.lipsum.com
2020-04-24 21:36:24,565 - DEBUG - http://et.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:24,569 - DEBUG - Starting new HTTPS connection (1): et.lipsum.com
2020-04-24 21:36:25,052 - DEBUG - https://et.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:25,061 - INFO - Good links written
2020-04-24 21:36:25,070 - DEBUG - Starting new HTTP connection (1): ph.lipsum.com
2020-04-24 21:36:25,421 - DEBUG - http://ph.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:25,426 - DEBUG - Starting new HTTPS connection (1): ph.lipsum.com
2020-04-24 21:36:25,905 - DEBUG - https://ph.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:25,914 - INFO - Good links written
2020-04-24 21:36:25,924 - DEBUG - Starting new HTTP connection (1): fi.lipsum.com
2020-04-24 21:36:26,272 - DEBUG - http://fi.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:26,277 - DEBUG - Starting new HTTPS connection (1): fi.lipsum.com
2020-04-24 21:36:26,767 - DEBUG - https://fi.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:26,778 - INFO - Good links written
2020-04-24 21:36:26,785 - DEBUG - Starting new HTTP connection (1): fr.lipsum.com
2020-04-24 21:36:27,139 - DEBUG - http://fr.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:27,144 - DEBUG - Starting new HTTPS connection (1): fr.lipsum.com
2020-04-24 21:36:27,626 - DEBUG - https://fr.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:27,634 - INFO - Good links written
2020-04-24 21:36:27,642 - DEBUG - Starting new HTTP connection (1): ka.lipsum.com
2020-04-24 21:36:28,007 - DEBUG - http://ka.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:28,011 - DEBUG - Starting new HTTPS connection (1): ka.lipsum.com
2020-04-24 21:36:28,500 - DEBUG - https://ka.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:28,510 - INFO - Good links written
2020-04-24 21:36:28,518 - DEBUG - Starting new HTTP connection (1): de.lipsum.com
2020-04-24 21:36:28,867 - DEBUG - http://de.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:28,871 - DEBUG - Starting new HTTPS connection (1): de.lipsum.com
2020-04-24 21:36:29,358 - DEBUG - https://de.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:29,367 - INFO - Good links written
2020-04-24 21:36:29,382 - DEBUG - Starting new HTTP connection (1): el.lipsum.com
2020-04-24 21:36:29,735 - DEBUG - http://el.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:29,740 - DEBUG - Starting new HTTPS connection (1): el.lipsum.com
2020-04-24 21:36:30,216 - DEBUG - https://el.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:30,225 - INFO - Good links written
2020-04-24 21:36:30,229 - DEBUG - Starting new HTTP connection (1): he.lipsum.com
2020-04-24 21:36:30,588 - DEBUG - http://he.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:30,596 - DEBUG - Starting new HTTPS connection (1): he.lipsum.com
2020-04-24 21:36:31,092 - DEBUG - https://he.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:31,102 - INFO - Good links written
2020-04-24 21:36:31,110 - DEBUG - Starting new HTTP connection (1): he.lipsum.com
2020-04-24 21:36:31,337 - DEBUG - http://he.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:31,342 - DEBUG - Starting new HTTPS connection (1): he.lipsum.com
2020-04-24 21:36:31,830 - DEBUG - https://he.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:31,840 - INFO - Good links written
2020-04-24 21:36:31,848 - DEBUG - Starting new HTTP connection (1): hi.lipsum.com
2020-04-24 21:36:32,200 - DEBUG - http://hi.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:32,205 - DEBUG - Starting new HTTPS connection (1): hi.lipsum.com
2020-04-24 21:36:32,687 - DEBUG - https://hi.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:32,697 - INFO - Good links written
2020-04-24 21:36:32,706 - DEBUG - Starting new HTTP connection (1): hu.lipsum.com
2020-04-24 21:36:33,053 - DEBUG - http://hu.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:33,057 - DEBUG - Starting new HTTPS connection (1): hu.lipsum.com
2020-04-24 21:36:33,538 - DEBUG - https://hu.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:33,548 - INFO - Good links written
2020-04-24 21:36:33,556 - DEBUG - Starting new HTTP connection (1): id.lipsum.com
2020-04-24 21:36:33,915 - DEBUG - http://id.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:33,925 - DEBUG - Starting new HTTPS connection (1): id.lipsum.com
2020-04-24 21:36:34,407 - DEBUG - https://id.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:34,418 - INFO - Good links written
2020-04-24 21:36:34,425 - DEBUG - Starting new HTTP connection (1): it.lipsum.com
2020-04-24 21:36:34,772 - DEBUG - http://it.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:34,779 - DEBUG - Starting new HTTPS connection (1): it.lipsum.com
2020-04-24 21:36:35,258 - DEBUG - https://it.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:35,268 - INFO - Good links written
2020-04-24 21:36:35,275 - DEBUG - Starting new HTTP connection (1): lv.lipsum.com
2020-04-24 21:36:35,628 - DEBUG - http://lv.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:35,633 - DEBUG - Starting new HTTPS connection (1): lv.lipsum.com
2020-04-24 21:36:36,115 - DEBUG - https://lv.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:36,124 - INFO - Good links written
2020-04-24 21:36:36,133 - DEBUG - Starting new HTTP connection (1): lt.lipsum.com
2020-04-24 21:36:36,482 - DEBUG - http://lt.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:36,486 - DEBUG - Starting new HTTPS connection (1): lt.lipsum.com
2020-04-24 21:36:36,979 - DEBUG - https://lt.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:36,989 - INFO - Good links written
2020-04-24 21:36:36,999 - DEBUG - Starting new HTTP connection (1): mk.lipsum.com
2020-04-24 21:36:37,373 - DEBUG - http://mk.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:37,378 - DEBUG - Starting new HTTPS connection (1): mk.lipsum.com
2020-04-24 21:36:37,866 - DEBUG - https://mk.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:37,876 - INFO - Good links written
2020-04-24 21:36:37,884 - DEBUG - Starting new HTTP connection (1): ms.lipsum.com
2020-04-24 21:36:38,230 - DEBUG - http://ms.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:38,234 - DEBUG - Starting new HTTPS connection (1): ms.lipsum.com
2020-04-24 21:36:38,718 - DEBUG - https://ms.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:38,727 - INFO - Good links written
2020-04-24 21:36:38,740 - DEBUG - Starting new HTTP connection (1): no.lipsum.com
2020-04-24 21:36:39,089 - DEBUG - http://no.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:39,094 - DEBUG - Starting new HTTPS connection (1): no.lipsum.com
2020-04-24 21:36:39,585 - DEBUG - https://no.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:39,594 - INFO - Good links written
2020-04-24 21:36:39,603 - DEBUG - Starting new HTTP connection (1): pl.lipsum.com
2020-04-24 21:36:39,952 - DEBUG - http://pl.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:39,957 - DEBUG - Starting new HTTPS connection (1): pl.lipsum.com
2020-04-24 21:36:40,442 - DEBUG - https://pl.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:40,452 - INFO - Good links written
2020-04-24 21:36:40,461 - DEBUG - Starting new HTTP connection (1): pt.lipsum.com
2020-04-24 21:36:40,810 - DEBUG - http://pt.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:40,815 - DEBUG - Starting new HTTPS connection (1): pt.lipsum.com
2020-04-24 21:36:41,307 - DEBUG - https://pt.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:41,313 - INFO - Good links written
2020-04-24 21:36:41,327 - DEBUG - Starting new HTTP connection (1): ro.lipsum.com
2020-04-24 21:36:41,691 - DEBUG - http://ro.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:41,696 - DEBUG - Starting new HTTPS connection (1): ro.lipsum.com
2020-04-24 21:36:42,179 - DEBUG - https://ro.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:42,189 - INFO - Good links written
2020-04-24 21:36:42,199 - DEBUG - Starting new HTTP connection (1): ru.lipsum.com
2020-04-24 21:36:42,544 - DEBUG - http://ru.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:42,549 - DEBUG - Starting new HTTPS connection (1): ru.lipsum.com
2020-04-24 21:36:43,029 - DEBUG - https://ru.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:43,040 - INFO - Good links written
2020-04-24 21:36:43,046 - DEBUG - Starting new HTTP connection (1): sr.lipsum.com
2020-04-24 21:36:43,400 - DEBUG - http://sr.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:43,404 - DEBUG - Starting new HTTPS connection (1): sr.lipsum.com
2020-04-24 21:36:43,886 - DEBUG - https://sr.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:43,896 - INFO - Good links written
2020-04-24 21:36:43,905 - DEBUG - Starting new HTTP connection (1): sk.lipsum.com
2020-04-24 21:36:44,288 - DEBUG - http://sk.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:44,293 - DEBUG - Starting new HTTPS connection (1): sk.lipsum.com
2020-04-24 21:36:44,776 - DEBUG - https://sk.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:44,785 - INFO - Good links written
2020-04-24 21:36:44,797 - DEBUG - Starting new HTTP connection (1): sl.lipsum.com
2020-04-24 21:36:45,143 - DEBUG - http://sl.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:45,147 - DEBUG - Starting new HTTPS connection (1): sl.lipsum.com
2020-04-24 21:36:45,639 - DEBUG - https://sl.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:45,650 - INFO - Good links written
2020-04-24 21:36:45,658 - DEBUG - Starting new HTTP connection (1): es.lipsum.com
2020-04-24 21:36:46,007 - DEBUG - http://es.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:46,012 - DEBUG - Starting new HTTPS connection (1): es.lipsum.com
2020-04-24 21:36:46,494 - DEBUG - https://es.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:46,503 - INFO - Good links written
2020-04-24 21:36:46,511 - DEBUG - Starting new HTTP connection (1): sv.lipsum.com
2020-04-24 21:36:46,868 - DEBUG - http://sv.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:46,872 - DEBUG - Starting new HTTPS connection (1): sv.lipsum.com
2020-04-24 21:36:47,356 - DEBUG - https://sv.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:47,365 - INFO - Good links written
2020-04-24 21:36:47,380 - DEBUG - Starting new HTTP connection (1): th.lipsum.com
2020-04-24 21:36:47,745 - DEBUG - http://th.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:47,749 - DEBUG - Starting new HTTPS connection (1): th.lipsum.com
2020-04-24 21:36:48,249 - DEBUG - https://th.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:48,253 - INFO - Good links written
2020-04-24 21:36:48,268 - DEBUG - Starting new HTTP connection (1): tr.lipsum.com
2020-04-24 21:36:48,617 - DEBUG - http://tr.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:48,621 - DEBUG - Starting new HTTPS connection (1): tr.lipsum.com
2020-04-24 21:36:49,110 - DEBUG - https://tr.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:49,120 - INFO - Good links written
2020-04-24 21:36:49,129 - DEBUG - Starting new HTTP connection (1): uk.lipsum.com
2020-04-24 21:36:49,481 - DEBUG - http://uk.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:49,487 - DEBUG - Starting new HTTPS connection (1): uk.lipsum.com
2020-04-24 21:36:49,966 - DEBUG - https://uk.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:49,976 - INFO - Good links written
2020-04-24 21:36:49,984 - DEBUG - Starting new HTTP connection (1): vi.lipsum.com
2020-04-24 21:36:50,333 - DEBUG - http://vi.lipsum.com:80 "GET / HTTP/1.1" 301 None
2020-04-24 21:36:50,338 - DEBUG - Starting new HTTPS connection (1): vi.lipsum.com
2020-04-24 21:36:50,824 - DEBUG - https://vi.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:50,834 - INFO - Good links written
2020-04-24 21:36:50,840 - DEBUG - Starting new HTTPS connection (1): www.lipsum.com
2020-04-24 21:36:51,314 - DEBUG - https://www.lipsum.com:443 "GET //banners HTTP/1.1" 301 239
2020-04-24 21:36:51,440 - DEBUG - https://www.lipsum.com:443 "GET /banners/ HTTP/1.1" 200 None
2020-04-24 21:36:51,445 - INFO - Good links written
2020-04-24 21:36:51,456 - DEBUG - Starting new HTTPS connection (1): www.lipsum.com
2020-04-24 21:36:51,925 - DEBUG - https://www.lipsum.com:443 "GET //banners HTTP/1.1" 301 239
2020-04-24 21:36:52,052 - DEBUG - https://www.lipsum.com:443 "GET /banners/ HTTP/1.1" 200 None
2020-04-24 21:36:52,057 - INFO - Good links written
2020-04-24 21:36:52,071 - DEBUG - Starting new HTTPS connection (1): www.lipsum.com
2020-04-24 21:36:52,536 - DEBUG - https://www.lipsum.com:443 "GET //banners HTTP/1.1" 301 239
2020-04-24 21:36:52,661 - DEBUG - https://www.lipsum.com:443 "GET /banners/ HTTP/1.1" 200 None
2020-04-24 21:36:52,664 - INFO - Good links written
2020-04-24 21:36:52,680 - DEBUG - Starting new HTTPS connection (1): www.lipsum.com
2020-04-24 21:36:53,145 - DEBUG - https://www.lipsum.com:443 "GET //banners HTTP/1.1" 301 239
2020-04-24 21:36:53,271 - DEBUG - https://www.lipsum.com:443 "GET /banners/ HTTP/1.1" 200 None
2020-04-24 21:36:53,274 - INFO - Good links written
2020-04-24 21:36:53,277 - DEBUG - Starting new HTTPS connection (1): www.lipsum.com
2020-04-24 21:36:53,768 - DEBUG - https://www.lipsum.com:443 "GET //donate HTTP/1.1" 200 None
2020-04-24 21:36:53,777 - INFO - Good links written
2020-04-24 21:36:53,785 - DEBUG - Starting new HTTPS connection (1): addons.mozilla.org
2020-04-24 21:36:55,010 - DEBUG - https://addons.mozilla.org:443 "GET /en-US/firefox/addon/dummy-lipsum/ HTTP/1.1" 404 16412
2020-04-24 21:36:55,012 - INFO - Bad links written
2020-04-24 21:36:55,017 - DEBUG - Starting new HTTPS connection (1): github.com
2020-04-24 21:36:56,211 - DEBUG - https://github.com:443 "GET /traviskaufman/node-lipsum HTTP/1.1" 200 None
2020-04-24 21:36:56,266 - INFO - Good links written
2020-04-24 21:36:56,270 - DEBUG - Starting new HTTP connection (1): ftp.ktug.or.kr
2020-04-24 21:36:57,224 - DEBUG - http://ftp.ktug.or.kr:80 "GET /tex-archive/help/Catalogue/entries/lipsum.html HTTP/1.1" 301 263
2020-04-24 21:36:57,230 - DEBUG - Starting new HTTP connection (1): ftp.ktug.org
2020-04-24 21:36:57,980 - DEBUG - http://ftp.ktug.org:80 "GET /tex-archive/help/Catalogue/entries/lipsum.html HTTP/1.1" 404 250
2020-04-24 21:36:57,982 - INFO - Bad links written
2020-04-24 21:36:57,986 - DEBUG - Starting new HTTP connection (1): code.google.com
2020-04-24 21:36:58,262 - DEBUG - http://code.google.com:80 "GET /p/pypsum/ HTTP/1.1" 301 287
2020-04-24 21:36:58,267 - DEBUG - Starting new HTTPS connection (1): code.google.com
2020-04-24 21:36:58,526 - DEBUG - https://code.google.com:443 "GET /archive/p/pypsum HTTP/1.1" 200 2759
2020-04-24 21:36:58,529 - INFO - Good links written
2020-04-24 21:36:58,543 - DEBUG - Starting new HTTP connection (1): gtklipsum.sourceforge.net
2020-04-24 21:36:59,063 - DEBUG - http://gtklipsum.sourceforge.net:80 "GET / HTTP/1.1" 200 None
2020-04-24 21:36:59,064 - INFO - Good links written
2020-04-24 21:36:59,069 - DEBUG - Starting new HTTP connection (1): github.com
2020-04-24 21:36:59,158 - DEBUG - http://github.com:80 "GET /gsavage/lorem_ipsum/tree/master HTTP/1.1" 301 0
2020-04-24 21:36:59,162 - DEBUG - Starting new HTTPS connection (1): github.com
2020-04-24 21:36:59,717 - DEBUG - https://github.com:443 "GET /gsavage/lorem_ipsum/tree/master HTTP/1.1" 200 None
2020-04-24 21:36:59,768 - INFO - Good links written
2020-04-24 21:36:59,771 - DEBUG - Starting new HTTPS connection (1): github.com
2020-04-24 21:37:00,080 - DEBUG - https://github.com:443 "GET /cerkit/LoremIpsum/ HTTP/1.1" 301 103
2020-04-24 21:37:00,545 - DEBUG - https://github.com:443 "GET /cerkit/lorem-ipsum HTTP/1.1" 200 None
2020-04-24 21:37:00,599 - INFO - Good links written
2020-04-24 21:37:00,604 - DEBUG - Starting new HTTP connection (1): groovyconsole.appspot.com
2020-04-24 21:37:00,838 - DEBUG - http://groovyconsole.appspot.com:80 "GET /script/64002 HTTP/1.1" 200 4881
2020-04-24 21:37:00,840 - INFO - Good links written
2020-04-24 21:37:00,858 - DEBUG - Starting new HTTP connection (1): www.layerhero.com
2020-04-24 21:37:00,944 - ERROR - Page Link Error: HTTPConnectionPool(host='www.layerhero.com', port=80): Max retries exceeded with url: /lorem-ipsum-generator/ (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))
2020-04-24 21:37:00,948 - DEBUG - Starting new HTTPS connection (1): www.lipsum.com
2020-04-24 21:37:01,417 - DEBUG - https://www.lipsum.com:443 "GET /mailto:help@lipsum.com HTTP/1.1" 302 None
2020-04-24 21:37:01,545 - DEBUG - https://www.lipsum.com:443 "GET / HTTP/1.1" 200 None
2020-04-24 21:37:01,553 - INFO - Good links written
2020-04-24 21:37:01,563 - DEBUG - Starting new HTTPS connection (1): www.lipsum.com
2020-04-24 21:37:02,025 - DEBUG - https://www.lipsum.com:443 "GET //privacy.pdf HTTP/1.1" 200 35352
2020-04-24 21:37:02,160 - INFO - Good links written

When I ran “python linkver.py “https://ajalacomfort.com”” I got the following links: