Catinsunshine's Blog

As more and more website use https instead of http, we'll encounter 403 error when get content with requests modules. To solve it, the easiest way is simulating browser to access. OS: Debian 9 Python: 3.5.3 Module: requests setup headers as 1st step: headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3)\            AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.81 Safari/537.36'} then add headers as parameter: res = requests.get(xqurl, headers=headers) Below is my example to get stock price in dictionary from a https website: getStockPriceFromXueqiu.py

To translate Domain name to IP address, there are 2 commands in Linux: 1. dig +short [Domain name] eg: $ dig +short google.com 216.58.197.142 2. nslookup [Domain name] eg:$ nslookup google.com Server:         172.31.0.2 Address:        172.31.0.2#53 Non-authoritative answer: Name:   google.com Address: 172.217.26.14

To check server current Internet speed, not the max speed in terminal/command line, the easiest way is speedometer. Take Ubuntu as example: Install: $ sudo apt install speedometer Usage: $ speedometer -rx eth0 unit conversion: bit=bit B=byte 1 byte=8 bit bps=bit/s MB/S = megabyte per second MiB/s = mebibyte per second 1 meibyte=2**20=1048576 bytes = 1024 kibibytes 1 Mbps=1 Mbit/s = 0.125MB/s =(0.125M/2**20)= 0.1192 MiB/s