Top 10 Tasks to Automate with Python

Automate with Python using the Python task automation scripts explained in this post. Download and install the “Top 10 Automatable Tasks” build, which contains a version of Python and all the tools listed in this post.

As a developer, there are always more tasks to complete than you have time for in a day. While it’s important to prioritize time-sensitive, impactful or blocking issues, they’re usually not the kind of things that can be knocked off your list quickly. If you try breaking down bigger tasks into smaller subtasks, you just end up increasing the list of things that still need to get done. But what if you could knock off some of these subtasks in no time at all? That’s where Python comes in.

Python is an incredibly powerful and flexible open-source language that is used in all layers of the application stack across the technology industry: from data manipulation processing real-time data feeds, to serverless applications processing commands from your personal voice assistant to being a rock-solid, general-purpose language for traditional applications.

Because of this flexibility and breadth of capabilities – extended by its extensive library of plugins – Python is an amazing tool. It can be used to build all the little scripts and utilities that simplify and automate with Python, your life as a software developer or system administrator.

Automate with Python: Getting Started

This post goes through a series of scripts that can be run on multiple platforms (Windows, Mac and Linux) to accomplish common tasks. To get started, you can:

1. Download and install the “Top 10 Automatable Tasks” build, which contains a version of Python and all the tools listed in this post so you can test them out for yourself:
   • Install the State Tool on Windows using Powershell:
     IEX(New-Object Net.WebClient).downloadString('https://platform.activestate.com/dl/cli/install.ps1')
   • Or install the State Tool on Linux / Mac:
     sh <(curl -q https://platform.activestate.com/dl/cli/install.sh)
2. Run the following command to download the build and automatically install it into a virtual environment:
   state activate Pizza-Team/Top-10-automatable-tasks

You can review all the relevant source code in my Python Scripts GitHub repo.

All set? Let’s go.

Top 10 Python Task Automation Scripts

#1 checkjson

This script will read a file and either pass the file as being a valid JSON file, or die a horrible death. But for all practical reasons, it tells me if the error is in the file or the program that I am trying to load the file into.

import os
import sys
import json
if len(sys.argv) > 1:
    if os.path.exists(sys.argv[1]):
        file = open(sys.argv[1], "r")
        json.load(file)
        file.close()
        print("Validate JSON!")
    else:
        print(sys.argv[1] + " not found")
else:
    print("Usage: checkjson.py <file>")

To run it:

$ checkjson test.json

That’s all you need to validate JSON, but it can easily be modified to validate YAML, as well.

#2 checkyaml

Kubernetes is difficult enough without having YAML fail to parse. This utility is a way to validate that the YAML is valid. Use it before ripping apart the program you are trying to load it into.

import os
import sys
import yaml
if len(sys.argv) > 1:
    if os.path.exists(sys.argv[1]):
        file = open(sys.argv[1], "r")
        yaml.safe_load(file.read())
        file.close()
        print("Validate YAML!")
    else:
        print(sys.argv[1] + " not found")
else:
    print("Usage: checkyaml.py <file>")

To run it:

$ checkyaml.py test.yaml

Now that we’ve validated that we’re dealing with JSON and YAML, how can we convert between them?

#3 json2yaml

If you work with config files or need to expose YAML via an API, you’ll probably find yourself needing to convert a file from JSON to YAML. The output can either be sent to stdout or to a specified file.

import json
import os
import sys
import yaml
# Checking there is a file name passed
if len(sys.argv) > 1:
    # Opening the file
    if os.path.exists(sys.argv[1]):
        source_file = open(sys.argv[1], "r")
        source_content = json.load(source_file)
        source_file.close()
    # Failikng if the file isn't found
    else:
        print("ERROR: " + sys.argv[1] + " not found")
        exit(1)
# No file, no usage
else:
    print("Usage: json2yaml.py <source_file.json> [target_file.yaml]")
# Processing the conversion
output = yaml.dump(source_content)
# If no target file send to stdout
if len(sys.argv) < 3:
    print(output)
# If the target file already exists exit
elif os.path.exists(sys.argv[2]):
    print("ERROR: " + sys.argv[2] + " already exists")
    exit(1)
# Otherwise write to the specified file
else:
    target_file = open(sys.argv[2], "w")
    target_file.write(output)
    target_file.close()

To run it:

$ json2yaml.py input_file.json output_file.yaml

But what if you need to go in the other direction, from YAML to JSON?

#4 yaml2json

This script converts a file from YAML to JSON. The output can either be sent to stdout or to a specified file.

import json
import os
import sys
import yaml
# Checking there is a file name passed
if len(sys.argv) > 1:
    # Opening the file
    if os.path.exists(sys.argv[1]):
        source_file = open(sys.argv[1], "r")
        source_content = yaml.safe_load(source_file)
        source_file.close()
    # Failikng if the file isn't found
    else:
        print("ERROR: " + sys.argv[1] + " not found")
        exit(1)
# No file, no usage
else:
    print("Usage: yaml2json.py <source_file.yaml> [target_file.json]")
# Processing the conversion
output = json.dumps(source_content)
# If no target file send to stdout
if len(sys.argv) < 3:
    print(output)
# If the target file already exists exit
elif os.path.exists(sys.argv[2]):
    print("ERROR: " + sys.argv[2] + " already exists")
    exit(1)
# Otherwise write to the specified file
else:
    target_file = open(sys.argv[2], "w")
    target_file.write(output)
    target_file.close()

To run it:

$ yaml2json.py input_file.yaml output_file.json

Manipulating JSON and YAML are great options for dealing with text input, but what if your input is a graphic file instead?

#5 convert2jpg

When dealing with older systems that require you to upload images, such as defect tracking applications or expense management solutions, it seems like they all accept JPEG images but rarely accept newer formats, such as PNG or HEIC.

To resolve this issue, here’s a quick utility that will take the input file and convert it to a .jpg version using the same base name.

import os
import sys
from PIL import Image
if len(sys.argv) > 1:
    if os.path.exists(sys.argv[1]):
        im = Image.open(sys.argv[1])
        target_name = sys.argv[1] + ".jpg"
        rgb_im = im.convert('RGB')
        rgb_im.save(target_name)
        print("Saved as " + target_name)
    else:
        print(sys.argv[1] + " not found")
else:
    print("Usage: convert2jpg.py <file>")

To run it:

$  convert2jpg test.png

Even more common than dealing with graphic files is copying and pasting info to/from the clipboard. But is there a way to do it without having to open an editor every time?

#6 copy2clip

Copying the contents of a text file to the clipboard is an all-too-frequent task, whether it’s a log file someone wants to see, a configuration file you need to share, or even an ssh key. Getting to the file is easy enough on the command line, but then having to open it in an editor just to copy-and-paste takes time you don’t need to waste.

The following copy2clip script works on both Windows and Mac, and uses native functionality that is wrapped in a little bit of Python code to load the files into the clipboard.

import os
import sys
import platform
import subprocess
# Seeing if the file exists
if os.path.exists(sys.argv[1]):
    f = open(sys.argv[1], "r")
    f_contents = f.read()
    f.close()
else:
    print("Usage: copy2clip <file_name>")
    exit(1)
whatos = platform.system()
if whatos == "Darwin":
    subprocess.run("pbcopy", universal_newlines=True, input=f_contents)
    print("success: copied to clipboard")
elif whatos == "Windows":
    subprocess.run("clip", universal_newlines=True, input=f_contents)
    print("success: copied to clipboard")
else:
    print("failed: clipboard not supported")

To run it:

$ copy2clip filename

So now that we know how to copy and paste file content quicker and easier, let’s apply it to file names.

#7 cwd

Getting the present working directory is easy on Linux and Mac because it’s built into the shell as the pwd command. But pwd is a POSIX environmental variable, which means it won’t work on Windows. As a result, we can use the command called cwd so it will not interfere if you choose to use it on either Mac or Linux, but is primarily aimed at Windows.

Running cwd on its own will display the current working directory. If you pass the -c flag it automatically copies the current working directory to the clipboard on Mac or Windows, which saves a lot of retyping and mouse movements.

In addition, you can pass a filename as an argument and have its full path copied to the clipboard, which is something I need to do often when running test cases.

import os
import sys
import platform
import subprocess
# No extra arguments are good
if len(sys.argv) == 1:
    copy_to_clipboard=False
    file_name = ""
# Checking if there is one extra argument
elif len(sys.argv) == 2:
    if sys.argv[1] == "-c":
        copy_to_clipboard=True
        file_name = ""
    elif os.path.exists(sys.argv[1]):
        file_name = sys.argv[1]
        copy_to_clipboard=False
    else:
        copy_to_clipboard=False
        file_name = ""
# Checking if there are two extra arguments
elif len(sys.argv) == 3:
    # Checking if the non-file value is set correctly
    if sys.argv[1] == "-c" or sys.argv[2] == "-c":
        copy_to_clipboard=True
    else:
        print("Usage: cwd [-c] [file_name]")
        exit(1)
    # Checking if the file exists before
    if os.path.exists(sys.argv[1]):
        file_name = sys.argv[1]
    elif os.path.exists(sys.argv[2]):
        file_name = sys.argv[2]
    else:
        print("Usage: cwd [-c] [file_name]")
        exit(1)
# More than two extra are bad
else:
    print("Usage: cwd [-c] [file_name]")
    exit(1)
# If file_name is setup then get abspath otherwise get cwd
if len(file_name) >= 1:
    full_path = os.path.abspath(file_name)
else:
    full_path = os.getcwd()
# If clipboard is setup add it to clipboard on Windows or Mac
if copy_to_clipboard == True:
    whatos = platform.system()
    if whatos == "Darwin":
        subprocess.run("pbcopy", universal_newlines=True, input=full_path)
        output = full_path + " (copied to clipboard)"
    elif whatos == "Windows":
        subprocess.run("clip", universal_newlines=True, input=full_path)
        output = full_path + " (copied to clipboard)"
    else:
        output = full_path + " (clipboard not supported)"
else:
    output = full_path
# Displaying output
print(output)

To run it:

user@macbook ~ % cwd.py -c cwd.bat
/Users/Demo/python-scripts/cwd.bat (copied to clipboard

Let’s change tack and talk math.

#8 quickmath

This script is used to add (or multiply or divide or subtract) multiple numbers in a single command line. The task of calculating values is needed more often than you think, and who wants to open Excel just for that? This automation comes in handy.

import math
import sys
if len(sys.argv) < 4:
    print("Usage: quickmath.py <add|mul|div|sub|avg> <num01> <num02> [num03..num99]")
    exit(1)
w_addition = {"+", "add", "addition", "plus"}
w_subtraction = {"-", "sub", "subtraction", "minus"}
w_multiply = {"*", "x", "mul", "multiply"}
w_divide = {"/", "div", "divide"}
for counter in range(2, len(sys.argv)):
    if not sys.argv[counter].isdigit():
        print("Usage: quickmath.py <add|mul|div|sub|avg> <num01> <num02> [num03..num99]")
        exit(1)
result = float(sys.argv[2])
if sys.argv[1] in w_addition:
    for counter in range(3, len(sys.argv)):
        result = result + float(sys.argv[counter])
elif sys.argv[1] in w_subtraction:
    for counter in range(3, len(sys.argv)):
        result = result - float(sys.argv[counter])
elif sys.argv[1] in w_multiply:
    for counter in range(3, len(sys.argv)):
        result = result * float(sys.argv[counter])
elif sys.argv[1] in w_divide:
    for counter in range(3, len(sys.argv)):
        result = result / float(sys.argv[counter])
else:
    print("Usage: quickmath.py <add|mul|div|sub|avg> <num01> <num02> [num03..num99]")
    exit(1)
print(str(result))

To run it:

$ quickmath add 2 10 18

Now that we can automate math, let’s apply it to currency conversion.

#9 currency

You’ve probably looked up conversion rates using your browser, but here’s how to automate the task of finding the current exchange rate between any two currencies.

import json
import sys
import urllib.request
if len(sys.argv) != 3:
    print("Usage: ./currencyrates.py lookup_currency base_currency. Example: ./currencyrates.py cad usd")
    sys.exit()
currency = sys.argv[1]
basecurrency = sys.argv[2]
currencyurl = "http://freecurrencyrates.com/api/action.php?do=cvals&iso=" + currency + "&f=" + basecurrency + "&v=1&s=cbr"
f = urllib.request.urlopen(currencyurl)
obj = json.loads(f.read())
result = "1 " + currency.upper() + " is "
result+="{:,.2f}".format(1/obj[currency.upper()]) + " " + basecurrency.upper()
print(result);

To run it:

$ currency cad usd

Now we know that $1 CDN is currently only worth $0.71 USD. Great for Americans; not so great for Canadians, but what about the other NATO countries?

#10 nato-alphabet

Okay, this one is just for fun. This script spells out a word using the NATO phonetic alphabet.

import sys
nato_alphabet = {
    'a': 'alpha',
    'b': 'bravo',
    'c': 'charlie',
    'd': 'delta',
    'e': 'echo',
    'f': 'foxtrot',
    'g': 'golf',
    'h': 'hotel',
    'i': 'india',
    'j': 'juliet',
    'k': 'kilo',
    'l': 'lima',
    'm': 'mike',
    'n': 'november',
    'o': 'oscar',
    'p': 'papa',
    'q': 'quebec',
    'r': 'romeo',
    's': 'sierra',
    't': 'tango',
    'u': 'uniform',
    'v': 'victor',
    'w': 'whiskey',
    'x': 'x-ray',
    'y': 'yankee',
    'z': 'zulu'
}
try:
    sys.argv[1]
except:
    print("Usage: natoalphabet.py <word>")
    exit(1)
for letter in sys.argv[1]:
    if letter.lower() not in nato_alphabet:
        print(letter)
    else:
        print(nato_alphabet[letter])

To run it:

$ natoalphabet automate
a - alpha
u - uniform
t - tango
o - oscar
m - mike
a - alpha
t - tango
e - echo

Automate with Python: Scripts for Every Developer’s Toolbox

My scripts just scratch the surface of what’s automatable with Python. If you have a function or routine task you want to automate, there are multiple resources online to aid in that pursuit, including:

• Python’s official documentation
• Python Package Index  which is the vast library of third party packages used by pip
• Automate the Boring Stuff, which is an entire book on the subject of automation with Python.

Next Steps:

• Review the scripts in my GitHub repository
• Sign up for a free ActiveState Platform account so you can download and install the Top 10 Automatable Tasks runtime in order to build your very own predictive models.

Related Blogs:

Top 10 Python Tools To Make a Developer’s Life Easier

Top 10 Python Packages with Examples

Top 10 Python Packages Every Developer Should Learn

Frequently Asked Questions