AI’s Impact on Software Developer Jobs

You often read that jobs will disappear due to the rapid development of AI, including those of software developers.

An Example of AI Takeover

In 2015 I worked in the automotive industry developing a Navigation Data Standard (NDS) compiler. It generates digital maps used mainly in cars. Because large amounts of data are involved, it was important to compress them as much as possible. I developed a Python script. It records traffic congestion during the day on Chinese highways. The data is saved in NDS format.

Applied Solution

The problem is similar to image data compression (photo). The sharpness and original colors are preserved as much as possible.

The k-means cluster algorithm can be applied for this purpose. It is an unsupervised Machine Learning algorithm which translates inputs into centroids, i.e. the closest points in Euclidian distance. This is an iterative process, in which the centroids change and the input points are re-assigned with each iteration. The algorithm always converges, but not necessarily in the optimal way

For the particular NDS problem we have the following parameters:

Input

• Number of clusters k.
• 2-dimensional array with n elements containing the speed on a road segment. It records data every quarter during the day. The sampling is done on a daily basis, so n = 96 * number of road segments.

Output

• The specified number (k) of centroids in a 2-dimensional array containing the newly calculated speeds. These are aligned on a quarter of the day and a road segment.
• 1- dimensional list containing n indexes. These are numbers varying from 0 to k-1 indicating how the initial elements are assigned to the centroids.
• Inertia: the sum of squared distances of samples to their closest centroid.

The implementation in Python is quite simple, as there is an API function available in sklearn.cluster. Nevertheless, it took quite some time to finish the job because of the next reasons:

• Amount of data: the script requires more than 16 GB of memory. A minimum of 64 GB was needed.
• Time: the multiprocessing feature in Python was applied to run the script in parallel for all the regions in China. Even then it took more than 1 day.
• Providing the input data in the correct format. It had to be converted from a Comma Separated Value (CSV) file to a 2-dimensional array of the form:

[[0, speed_1],[1, speed_2],[],[n-1, speed_n]]

• Processing the output data of the form:

[[i0, new_speed_1],[i1, new_speed_2],[],[ik-1, new_speed_k]]

• Tweaking: find the k with a reasonable low inertia, but still with enough compression to meet the NDS standard.
• Providing initial centroids for optimal convergence.

Conclusion

Ten years later AI would have assisted me in writing the code, but there is still plenty of work that needs to be done manually by the programmer:

• Setting up the correct hardware configuration and tweak for optimal usage of available cores.
• Providing input data in the appropriate format, handling of output data and setting up the correct initialization for the algorithm.
• Testing: this is done in a highly automated way nowadays using Continuous Integration (CI). Yet, software test engineers are needed to set up and maintain test environments.

Average coding will disappear with AI. The emphasis will increasingly be on hardware and software design. The work will focus on the implementation of the input and output requirements, visual inspection and (automated) testing. Of course, code reviews will also remain necessary. All in all, no worries for programmers’ jobs.