Automate with Python using the Python task automation scripts explained in this post. Download and install the “Top 10 Automatable Tasks” build, which contains a version of Python and all the tools listed in this post.
As a developer, there are always more tasks to complete than you have time for in a day. While it’s important to prioritize time-sensitive, impactful or blocking issues, they’re usually not the kind of things that can be knocked off your list quickly. If you try breaking down bigger tasks into smaller subtasks, you just end up increasing the list of things that still need to get done. But what if you could knock off some of these subtasks in no time at all? That’s where Python comes in.
Python is an incredibly powerful and flexible open-source language that is used in all layers of the application stack across the technology industry: from data manipulation processing real-time data feeds, to serverless applications processing commands from your personal voice assistant to being a rock-solid, general-purpose language for traditional applications.
Because of this flexibility and breadth of capabilities – extended by its extensive library of plugins – Python is an amazing tool. It can be used to build all the little scripts and utilities that simplify and automate with Python, your life as a software developer or system administrator.
Automate with Python: Getting Started
This post goes through a series of scripts that can be run on multiple platforms (Windows, Mac and Linux) to accomplish common tasks. To get started, you can:
- Download and install the “Top 10 Automatable Tasks” build, which contains a version of Python and all the tools listed in this post so you can test them out for yourself:
- Install the State Tool on Windows using Powershell:
IEX(New-Object Net.WebClient).downloadString('https://platform.activestate.com/dl/cli/install.ps1')
- Or install the State Tool on Linux / Mac:
sh <(curl -q https://platform.activestate.com/dl/cli/install.sh)
- Install the State Tool on Windows using Powershell:
- Run the following command to download the build and automatically install it into a virtual environment:
state activate Pizza-Team/Top-10-automatable-tasks
You can review all the relevant source code in my Python Scripts GitHub repo.
All set? Let’s go.
Top 10 Python Task Automation Scripts
#1 checkjson
This script will read a file and either pass the file as being a valid JSON file, or die a horrible death. But for all practical reasons, it tells me if the error is in the file or the program that I am trying to load the file into.
import os import sys import json if len(sys.argv) > 1: if os.path.exists(sys.argv[1]): file = open(sys.argv[1], "r") json.load(file) file.close() print("Validate JSON!") else: print(sys.argv[1] + " not found") else: print("Usage: checkjson.py <file>")
To run it:
$ checkjson test.json
That’s all you need to validate JSON, but it can easily be modified to validate YAML, as well.
#2 checkyaml
Kubernetes is difficult enough without having YAML fail to parse. This utility is a way to validate that the YAML is valid. Use it before ripping apart the program you are trying to load it into.
import os import sys import yaml if len(sys.argv) > 1: if os.path.exists(sys.argv[1]): file = open(sys.argv[1], "r") yaml.safe_load(file.read()) file.close() print("Validate YAML!") else: print(sys.argv[1] + " not found") else: print("Usage: checkyaml.py <file>")
To run it:
$ checkyaml.py test.yaml
Now that we’ve validated that we’re dealing with JSON and YAML, how can we convert between them?
#3 json2yaml
If you work with config files or need to expose YAML via an API, you’ll probably find yourself needing to convert a file from JSON to YAML. The output can either be sent to stdout or to a specified file.
import json import os import sys import yaml # Checking there is a file name passed if len(sys.argv) > 1: # Opening the file if os.path.exists(sys.argv[1]): source_file = open(sys.argv[1], "r") source_content = json.load(source_file) source_file.close() # Failikng if the file isn't found else: print("ERROR: " + sys.argv[1] + " not found") exit(1) # No file, no usage else: print("Usage: json2yaml.py <source_file.json> [target_file.yaml]") # Processing the conversion output = yaml.dump(source_content) # If no target file send to stdout if len(sys.argv) < 3: print(output) # If the target file already exists exit elif os.path.exists(sys.argv[2]): print("ERROR: " + sys.argv[2] + " already exists") exit(1) # Otherwise write to the specified file else: target_file = open(sys.argv[2], "w") target_file.write(output) target_file.close()
To run it:
$ json2yaml.py input_file.json output_file.yaml
But what if you need to go in the other direction, from YAML to JSON?
#4 yaml2json
This script converts a file from YAML to JSON. The output can either be sent to stdout or to a specified file.
import json import os import sys import yaml # Checking there is a file name passed if len(sys.argv) > 1: # Opening the file if os.path.exists(sys.argv[1]): source_file = open(sys.argv[1], "r") source_content = yaml.safe_load(source_file) source_file.close() # Failikng if the file isn't found else: print("ERROR: " + sys.argv[1] + " not found") exit(1) # No file, no usage else: print("Usage: yaml2json.py <source_file.yaml> [target_file.json]") # Processing the conversion output = json.dumps(source_content) # If no target file send to stdout if len(sys.argv) < 3: print(output) # If the target file already exists exit elif os.path.exists(sys.argv[2]): print("ERROR: " + sys.argv[2] + " already exists") exit(1) # Otherwise write to the specified file else: target_file = open(sys.argv[2], "w") target_file.write(output) target_file.close()
To run it:
$ yaml2json.py input_file.yaml output_file.json
Manipulating JSON and YAML are great options for dealing with text input, but what if your input is a graphic file instead?
#5 convert2jpg
When dealing with older systems that require you to upload images, such as defect tracking applications or expense management solutions, it seems like they all accept JPEG images but rarely accept newer formats, such as PNG or HEIC.
To resolve this issue, here’s a quick utility that will take the input file and convert it to a .jpg version using the same base name.
import os import sys from PIL import Image if len(sys.argv) > 1: if os.path.exists(sys.argv[1]): im = Image.open(sys.argv[1]) target_name = sys.argv[1] + ".jpg" rgb_im = im.convert('RGB') rgb_im.save(target_name) print("Saved as " + target_name) else: print(sys.argv[1] + " not found") else: print("Usage: convert2jpg.py <file>")
To run it:
$ convert2jpg test.png
Even more common than dealing with graphic files is copying and pasting info to/from the clipboard. But is there a way to do it without having to open an editor every time?
#6 copy2clip
Copying the contents of a text file to the clipboard is an all-too-frequent task, whether it’s a log file someone wants to see, a configuration file you need to share, or even an ssh key. Getting to the file is easy enough on the command line, but then having to open it in an editor just to copy-and-paste takes time you don’t need to waste.
The following copy2clip script works on both Windows and Mac, and uses native functionality that is wrapped in a little bit of Python code to load the files into the clipboard.
import os import sys import platform import subprocess # Seeing if the file exists if os.path.exists(sys.argv[1]): f = open(sys.argv[1], "r") f_contents = f.read() f.close() else: print("Usage: copy2clip <file_name>") exit(1) whatos = platform.system() if whatos == "Darwin": subprocess.run("pbcopy", universal_newlines=True, input=f_contents) print("success: copied to clipboard") elif whatos == "Windows": subprocess.run("clip", universal_newlines=True, input=f_contents) print("success: copied to clipboard") else: print("failed: clipboard not supported")
To run it:
$ copy2clip filename
So now that we know how to copy and paste file content quicker and easier, let’s apply it to file names.
#7 cwd
Getting the present working directory is easy on Linux and Mac because it’s built into the shell as the pwd command. But pwd is a POSIX environmental variable, which means it won’t work on Windows. As a result, we can use the command called cwd so it will not interfere if you choose to use it on either Mac or Linux, but is primarily aimed at Windows.
Running cwd on its own will display the current working directory. If you pass the -c flag it automatically copies the current working directory to the clipboard on Mac or Windows, which saves a lot of retyping and mouse movements.
In addition, you can pass a filename as an argument and have its full path copied to the clipboard, which is something I need to do often when running test cases.
import os import sys import platform import subprocess # No extra arguments are good if len(sys.argv) == 1: copy_to_clipboard=False file_name = "" # Checking if there is one extra argument elif len(sys.argv) == 2: if sys.argv[1] == "-c": copy_to_clipboard=True file_name = "" elif os.path.exists(sys.argv[1]): file_name = sys.argv[1] copy_to_clipboard=False else: copy_to_clipboard=False file_name = "" # Checking if there are two extra arguments elif len(sys.argv) == 3: # Checking if the non-file value is set correctly if sys.argv[1] == "-c" or sys.argv[2] == "-c": copy_to_clipboard=True else: print("Usage: cwd [-c] [file_name]") exit(1) # Checking if the file exists before if os.path.exists(sys.argv[1]): file_name = sys.argv[1] elif os.path.exists(sys.argv[2]): file_name = sys.argv[2] else: print("Usage: cwd [-c] [file_name]") exit(1) # More than two extra are bad else: print("Usage: cwd [-c] [file_name]") exit(1) # If file_name is setup then get abspath otherwise get cwd if len(file_name) >= 1: full_path = os.path.abspath(file_name) else: full_path = os.getcwd() # If clipboard is setup add it to clipboard on Windows or Mac if copy_to_clipboard == True: whatos = platform.system() if whatos == "Darwin": subprocess.run("pbcopy", universal_newlines=True, input=full_path) output = full_path + " (copied to clipboard)" elif whatos == "Windows": subprocess.run("clip", universal_newlines=True, input=full_path) output = full_path + " (copied to clipboard)" else: output = full_path + " (clipboard not supported)" else: output = full_path # Displaying output print(output)
To run it:
user@macbook ~ % cwd.py -c cwd.bat /Users/Demo/python-scripts/cwd.bat (copied to clipboard
Let’s change tack and talk math.
#8 quickmath
This script is used to add (or multiply or divide or subtract) multiple numbers in a single command line. The task of calculating values is needed more often than you think, and who wants to open Excel just for that? This automation comes in handy.
import math import sys if len(sys.argv) < 4: print("Usage: quickmath.py <add|mul|div|sub|avg> <num01> <num02> [num03..num99]") exit(1) w_addition = {"+", "add", "addition", "plus"} w_subtraction = {"-", "sub", "subtraction", "minus"} w_multiply = {"*", "x", "mul", "multiply"} w_divide = {"/", "div", "divide"} for counter in range(2, len(sys.argv)): if not sys.argv[counter].isdigit(): print("Usage: quickmath.py <add|mul|div|sub|avg> <num01> <num02> [num03..num99]") exit(1) result = float(sys.argv[2]) if sys.argv[1] in w_addition: for counter in range(3, len(sys.argv)): result = result + float(sys.argv[counter]) elif sys.argv[1] in w_subtraction: for counter in range(3, len(sys.argv)): result = result - float(sys.argv[counter]) elif sys.argv[1] in w_multiply: for counter in range(3, len(sys.argv)): result = result * float(sys.argv[counter]) elif sys.argv[1] in w_divide: for counter in range(3, len(sys.argv)): result = result / float(sys.argv[counter]) else: print("Usage: quickmath.py <add|mul|div|sub|avg> <num01> <num02> [num03..num99]") exit(1) print(str(result))
To run it:
$ quickmath add 2 10 18
Now that we can automate math, let’s apply it to currency conversion.
#9 currency
You’ve probably looked up conversion rates using your browser, but here’s how to automate the task of finding the current exchange rate between any two currencies.
import json import sys import urllib.request if len(sys.argv) != 3: print("Usage: ./currencyrates.py lookup_currency base_currency. Example: ./currencyrates.py cad usd") sys.exit() currency = sys.argv[1] basecurrency = sys.argv[2] currencyurl = "http://freecurrencyrates.com/api/action.php?do=cvals&iso=" + currency + "&f=" + basecurrency + "&v=1&s=cbr" f = urllib.request.urlopen(currencyurl) obj = json.loads(f.read()) result = "1 " + currency.upper() + " is " result+="{:,.2f}".format(1/obj[currency.upper()]) + " " + basecurrency.upper() print(result);
To run it:
$ currency cad usd
Now we know that $1 CDN is currently only worth $0.71 USD. Great for Americans; not so great for Canadians, but what about the other NATO countries?
#10 nato-alphabet
Okay, this one is just for fun. This script spells out a word using the NATO phonetic alphabet.
import sys nato_alphabet = { 'a': 'alpha', 'b': 'bravo', 'c': 'charlie', 'd': 'delta', 'e': 'echo', 'f': 'foxtrot', 'g': 'golf', 'h': 'hotel', 'i': 'india', 'j': 'juliet', 'k': 'kilo', 'l': 'lima', 'm': 'mike', 'n': 'november', 'o': 'oscar', 'p': 'papa', 'q': 'quebec', 'r': 'romeo', 's': 'sierra', 't': 'tango', 'u': 'uniform', 'v': 'victor', 'w': 'whiskey', 'x': 'x-ray', 'y': 'yankee', 'z': 'zulu' } try: sys.argv[1] except: print("Usage: natoalphabet.py <word>") exit(1) for letter in sys.argv[1]: if letter.lower() not in nato_alphabet: print(letter) else: print(nato_alphabet[letter])
To run it:
$ natoalphabet automate a - alpha u - uniform t - tango o - oscar m - mike a - alpha t - tango e - echo
Automate with Python: Scripts for Every Developer’s Toolbox
My scripts just scratch the surface of what’s automatable with Python. If you have a function or routine task you want to automate, there are multiple resources online to aid in that pursuit, including:
- Python’s official documentation
- Python Package Index which is the vast library of third party packages used by pip
- Automate the Boring Stuff, which is an entire book on the subject of automation with Python.
Next Steps:
- Review the scripts in my GitHub repository
- Sign up for a free ActiveState Platform account so you can download and install the Top 10 Automatable Tasks runtime in order to build your very own predictive models.
Related Blogs:
Frequently Asked Questions
What tasks can you automate with Python?
The tasks you can automate with Python are limitless, but common tasks include reading/writing files, scraping data from web sites, sending emails/texts, interacting with APIs, updating spreadsheets, filling online forms, and much more.
Which Python packages are useful for Python automation scripts?
The tasks you should automate with Python include any task you have to do more than once. However, recurring tasks must be automated if you hope to keep up with your deliverables.
Despite the many different use cases for programming in Python, there are several packages that are especially useful above all, such as Pip, Python-dateutil, Requests, Docutils, Setuptools and PyTest.
What are the top tasks for Python automation?
The daily tasks that should be automated with Python largely depends on your job, but some common tasks worth automating include reminder emails/texts, managing files, scheduling tasks, manipulating images and working with CSV, JSON or spreadsheets.
This blog post provides the top 10 Python automation scripts that are useful for these and many other tasks. These scripts include checkjson, checkyaml, json2yaml, yaml2json, convert2jpg, copy2clip, cwd, quickmath and currency.
Get the “Top 10 Automatable Tasks” Python build, which includes the tools listed in this post.
Can Python be used to automate Windows tasks?
Python can be used to automate Windows system administration tasks. Python provides a number of packages and scripts that can help you automate many of the most common Windows IT tasks.