One of these two is getting smarter all the time ...
There are a number of theories for why we've had a divergence between wages and productivity in the past forty years. Some point to technology, others to the fact that there is now a massive "reserve army" of labor as China, India, and other large populations previously shut out of our international economy have joined it in force. There's probably some combination of both as jobs that can't be shipped overseas become the target for innovation while those that can be sent abroad are because that's the easiest thing to do. Either way, the majority of classes of labor have lost leverage in negotiating a greater return while capital has gained greatly.
As an engineer, I of course have an interest in the technology part of the equation. I have been thinking about this for a while and wanted to share those thoughts. I think the world has the potential for more radical changes than we may realize. I'm thinking there are four parts to this tour which can be broken out as follows:
- Where do the machines' powers come from?
- What might we expect to see machines able to do?
- Worse than cyberpunk: Software sorcerers and skill-less serfs - the dark path
- Paradise found: Collective Cornucopias - the bright path
Part 1 and 2 of course sets the range of travel for parts 3 and 4. I'm sure those more socially inclined than me might be able to extrapolate the cultural bits more accurately than me. But I'm still going to have fun doing it.
As a note, if I throw around terms like “imperative programming” in this Part, expect to have them explained in Part 1.
Part 2: What might we expect machines to be able to do?
Welcome back. I'm especially happy that this means you weren't too bored by my first written lecture. I wanted to set down a lot of necessary background before the speculation begins. That job done, we can start to get to the fun part.
Diving in, we can first look at what machines can do today. It's pretty impressive actually. Anything that you would ask a person to do from a checklist is already in scope for computers. Compile a mailing list of Bernie supporters that contributed more than $200 so far? Done. Set up and run a test on some industrial equipment? Totally doable once the right wires are attached. You can just see the rafts of stevedores, clerks, accountants, order takers, and so on being displaced. Those remaining are now much more like foremen, checking results, directing the action, and deciding how the rules may need to be updated.
Assembly lines are obviously able to be automated and many of them are. If you can break a task down into pieces small enough to be subjected to time studies, chances are good that it can be fed to a computer controller as well. The instrumentation to feed the program and check on the state of the machine is often as complicated or more so than providing the instructions. Bugs come as often from sensor breakdowns or misinterpretation of their inputs as anything else. The main limitation here is that the exquisite dexterity and versatility of human manual manipulation is still ahead of the machines, which confines industrial automation to only those tasks done in huge numbers like in the automotive industry.
A good quality to look for in a task for automation in the current range of machine capability is whether the next action can be entirely determined from the previous one(s). That makes the specification and the checking very straightforward. If you need to form a high-level strategy or do a lot of planning ahead, that becomes a lot harder to describe as a checklist and thus less approachable in the imperative programming paradigm.
The next big steps forward in “intelligence”
That's today of course. Now we can duck into our bag of tricks from Part 1 and see what kinds of new things could be done. As described there, we’ve been spending a lot of the past few years in software development making tools that lets us ask machines to do things in ways that are more like how we’d ask each other. We are moving into the world of what is currently called “soft AI.” This type of AI is not a general intelligence like you or me, but it can start to do a number of things that are “smarter” than rote rule-following. Personally, until we see something pass general intelligence tests, we’ll start to see “soft AI” become more and more a stand-in for “intelligence tasks we know how to do or almost know how” versus stuff we haven’t figured out yet.
Matching problems to solutions
The first big help for the machine is automatic classification. Think of Facebook or any other website where you can upload files. Nine times out of ten it will ask what it is you just added, whether through tags (the more flexible option) or metadata forms (the one you have to do the moment you upload or never). Imagine doing this on an industrial scale or maybe you actually do. I assign tasks to my team and receive them from others in a bug tracking system and the addition of required metadata is a huge pain. Every upload to our document store requires me to describe the relative importance, general content, file type, level of officialness, and so on. Doing only a couple of these a day is annoying; trying to do a couple thousand would be soul-sucking. And admittedly, I do a terrible job of it — I’m sure the people studying what I’ve done hate me for my paltry contributions.
Not only does automated classification save some drudgery in archive management, it also provides a big lever arm in increasing the flexibility of machine programs. Think about how you would describe "picture of Aunt Linda" to a computer. The code would be intricate and strange. It’s really hard to look at an array of pixels and form the kind of high-level understanding needed to make that connection. Through the magic of automated classification and a large enough data set, computers can now recognize Aunt Linda. Maybe they are told that a nice thing to do is find a picture with you and her together from a couple of years ago and send her a reminder link on her birthday. Maybe she'll find it creepy, but the key here is that we're moving from the domain of meticulous programming to much higher-level instruction.
Moving back to the economic realm, let's connect classification to some other parts of our bag of tricks. Let's say I now want the computer to start setting up to plow my field for some corn. The computer recognizes your field as a plot of land with a given perimeter. It knows now that "fields" have soil and are exposed to weather and are primarily set up to crow crops, of which corn is one. It also knows that "plow" is an action where a given machine is taken over the field as a kind of preparation.
The classification problem is now combined with a specification of intent: assure that the entire field has seen the plow once let's say. Now this rule is itself classified - it's a problem where you want a linear path with a width such that a surface is efficiently covered. We now go the library and find such an algorithm and transition over into search space. The variables of number and direction of turns is introduced with distances between them. Once optimized, the route can be converted into controller code for your robo-tractor to set up and get going. If you want to get really fancy, the machine can look at fuel consumption and time curves for your tractor and estimate the amount of time the job takes and how much fuel you'll need to buy to complete it.
What I've just described is a big deal in programming efficiency. Rather than painstakingly measuring the field, understanding the steering qualities of the tractor, and hand-designing and laying in a course, we've just asked for a basic job to be done and this soft AI has filled in almost all of the blanks. The parts that contributed to its answer are highly reusable and can be stored in a common place for later use.
The type of approach I just described may be to intellectual work as the assembly line was to craft work. In other words, efficient in a way that is devastating to the autonomy and uniqueness of the job-doer but a massive savings to the job-buyer. In the current programming regime, it probably would have taken you a team of at least a couple of programmers, copious data gathering and modeling, and probably several months at the very least to make your robo-tractor go. If the right libraries are available as I described above, you might set this up the same way you order a pizza online today.
Actually, if you want a good example of the paint-by-numbers style of putting together problems and tools into useful programs, look at the growing malware industry (yes, I said industry). Basically, low-skill hackers have access to pre-made kits of code that can be assembled into working viruses. It’s so easy, tech journalists can do it. This kind of thing can really take off with a little bit more help from the computer to make helpful suggestions just like it does now with Google searches.
We aren't at this level of sophistication just yet but it isn't that far away. The big trick is in figuring out how to build all the connections between the steps I described. How does the general domain of tractors get connected to the general domains of path planning? It can start manually, since just linking ideas together is a lot less time-intensive than writing all the code by hand (or even stitching libraries together). But eventually the libraries will be big enough to make even this step easier and easier to achieve.
Here are a series of libraries I can see as being key to making swaths of automation jobs available to the next increment of soft AI. Note that a lot of this about reaching out to find information from different sets and stitching them together. We’ll talk about how how the sets are built shortly.
- Media to concept recognition: Speech, text, video (in visual or other spectral light), smell, tactile
- Concept and relationship dictionary, particularly economically useful actions (e.g., plow, drive, apply manufacturing step)
- Library of actions that can have parameters set (e.g., cover area within parameter by swaths of given width by some vehicle)
- Libraries to find / calculate parameters for actions (e.g., looking up your property boundaries and effective swatch for tractor plow)
- Real-world modeling libraries to connect to sensors including infrastructural sensors like GPS
Here I've come to the central plot. Basic research in the physical sciences and applied research in engineering churns out models and techniques to be used by experienced designers and operators to get ever more efficient in increasing our mechanized physical prowess. Building up vocabularies and libraries of software allows experts to get progressively more efficient at building even more software. This is an exponential growth, which means the rate of change will likely only increase and we will be surprised by how rapidly the future arrives from an unexpected direction.
Kinesthetic learners
Deep learning on huge amounts of data isn’t just for Google any more. Similar approaches are being taken to support robotic development. There is an active data-gathering experiment at the Cornell robotics lab called “Tell Me Dave” intended to crowd-source the gathering of data needed to teach the robot to be more dexterous. In addition, the work is already in progress to make the connections to the results of classification that I describe above. You can watch how tasks are being done on the same website.
The same group is teaching their robots how to test unknown foods for different properties like hardness or how stretchy it is. These properties change how you have to slice, move, etc. the bits of food (think of slicing hard cheese versus bread). The robot is shown in a video learning about tofu and what to do with it, as well as being able to prepare a salad. Of course, the robot is currently not anything like what I would call dexterous. People with severe arthritis can probably still out-race this machine at its tasks. However, once things are learned by one machine to get faster, the knowledge can spread pretty easily.
Of course, there are the driverless car projects. In true Google fashion, their approach is to gather huge amounts of data and then crunch it to understand how the various sensors can translate the outside world into internal models that can be manipulated to create new actions. This kind of research will probably take off soon as Uber has basically kidnapped the highly successful CMU driverless car team to pump full of Sillicon Valley money.
As for getting around on natural ground, there is obviously a lot of effort in this direction as well. Bipedal motion is still a very hard problem, so many of these platforms look more like insects or various other animals. There is an interesting set of choices to make here. The key to human manipulation was to become bipedal. However, we were constrained by evolution to not be able to change our physical form too much. Our ancestors had four limbs so we did too. If we wanted two of them to be made free for toolmaking and manipulation of the physical world, we had to rear up on the back two. There’s no reason to think that walking on these hind legs is really an optimal solution (and many reasons to think it is not) but we were forced to make do. Obviously we could invent robots with low centers of gravity for the mobility platform and then have a mast with a couple of arms hanging off of it as the manipulators.
making data for yourself
There is a wild card here I have yet to describe: experimentation. People learn through experimentation all the time of course. There are now automated laboratories that can select growth media, treatment drugs and various bacteria to grow. In another thread, there is a lot of work going on now with "deep learning" where samples of various media are given to computers to categorize. This can include finding the subjects in pictures, distinguishing people from backgrounds, extracting appropriate bits of language from common usage and so on. It is not unreasonable to expect that these two abilities can be combined eventually. If there are enough data to classify confidently, go ahead and classify. If not, try a little experimentation where things seem fuzzy. From these efforts can come both models and classifications. The Cornell robotics lab seems to already be doing this with its chef robot.
The experimentation capability may allow machines to brute force their way through the "too open-ended" question. Through simulated or real experiments, machines could start to work out some high-order strategies to big problems. This is not to say they may magically become effective. Like with people, there is a strong "garbage in, garbage out" factor to which forming strategies based on models are subject. Take your favorite Republican's belief in how motivation works vis-a-vis assistance for an example of how horribly wrong things can go. Or homo economus being destroyed in the laboratory. So if the simulations are bad, the strategies that are developed from playing with them will also be bad. That means the simulations will also need new data to improve.
With the door opened by experimentation and some brute-force model fitting, I would suggest that only the most uniquely human tasks become unavailable to machines. These are tasks which either require high creativity (creativity which goes beyond basic variation and choice) or need other humans to provide feedback on when the job is done. In the service industry, for example, it is easy for a floor salesperson to understand how satisfied you are with a purchase. It is very hard for machines to do the same. Of course, with enough data, who knows?
What the next wave of automation looks like
Looking at all of this, the next wave of tasks to be computerized appear to have the following characteristics:
Can be mathematically or logically modeled. There is a physical or logical reality surrounding the task that can be accurately modeled over a wide range of operating conditions. The requirements of modeling accuracy may be reduced greatly by sensor feedback if the things being sensed connect to the model.
Has a well-defined goal. This is probably the key point. The task needs to have a "am I done yet?" check for the machine to know when it is done. The more concrete and measurable, the better. So if the goal is "get these three items down the chute" that's a good one.
Can be described within relationships to abstract objects. Classifications make it easier to understand what the essence of the task is and connect it to ways to perform it. As described above, there is a huge amount of leverage in this attribute to expand the kinds of tasks we can ask for. The old way of specifying a route might be "take me from point A (my house) to point B (some venue)" is much more concrete than "take me from here to the concert an hour early." The concert is an event which takes place at a venue at a given place. From these relationships we can calculate time and place to drop you off.
Is not too open-ended. A lot of solution approaches still require the computer to find the right values to plug into a set of parameters on the program. If the possibilities are too numerous, the program will not finish quickly enough to be useful. Plotting a route, choosing a chess move, or identifying a cancer cell in an image isn't too expansive. Planning all the ways a full political campaign can evolve is way too large to be solved. There are certainly limits in this area to what can really be done - this is where the unique ingenuity of people in abstracting and strategizing really shines over what intelligence machines currently have. On the other hand, maybe it’s just a matter of finding the right simulation to experiment with …
Requires mobility and manipulation in this robotics generation or the next couple of them. Bipedal humans using their hands engage in an exquisite dance of balance and employ remarkable dexterity. There are increasing numbers of robotics companies attacking this problem head-on and others using creative form factors to side step it (see Robo-Simian for example). It's probably coming in the same wave as the others but currently is not cheap. As with all other technologies, it's just a matter of time and compiling the right tricks and recipes. There's probably some magical price-point at which we see these things really take off.
So what we see here is a transition from the totally defined problems that current computers are good at to the fairly-well-defined-but-you-fill-in-some-gaps kind of problems.
In the great summation, what can machines do that would displace humans in the economy? Let's go down the list using the Bureau of Labor Statistic's SOC classification system:
11-0000 Management Occupations
This category contains some of the jobs most resistance to machine substitution (or is that what the bosses want you to think?) due to the many human interactions. However, there are a lot of "management" jobs that are glorified logistics, which the machines will be quite good at. So there will likely be some shocking headlines like "meet HAL, your new boss" when it comes to thinks like tracking work done versus hours used.
13-0000 Business and Financial Operations
Yup, more logistics. The many tools of modeling, classification, and optimization will make it harder and harder for people to compete. Lots of medium-skilled, decently-paid occupations here that will be under pressure.
15-0000 Computer and Mathematical Occupations
Another zone of white-collar jobs were machines may make surprisingly deep inroads. As libraries of canned routines become available, software developers will have to become ever more clever to hold an advantage. The User Experience world, with a need to deeply understand people, will be one of the safer havens without an advanced degree. Ditto mathematics where computerized techniques are becoming increasingly sophisticated. Many proofs can now be automatically generated on pure symbols using systems like Wolfram Research's Mathematica.
17-0000 Architecture and Engineering Occupations
I'm part of the problem on this one. Again, we may be surprised on how deep machine intelligence can go into complexity for engineering problems. Take Edison's lightbulb for example - the winning formula was a lot more perspiration (trying lots of combinations of bulb and filament) than inspiration. I know that I can amplify my own abilities through computation. Once we get a better hang of 3d printing and rapid instrumentation for real-world experiments, we will likely see a lot of machine augmentation of engineers and smaller staffs.
19-0000 Life, Physical, and Social Science Occupations
Lots of creativity and inventiveness needed here to design and implement experiments. But as with the other white collars, expect to be surprised just how much "brute force" methods employed by the machines will be able to achieve.
21-0000 Community and Social Service Occupations
These jobs are intensely human and require a lot of empathy and connection. I find it hard to believe machines will make great inroads here. Of course, that is when the jobs are performed at their best. Given our nation’s typical under-appreciation of this kind of work, expect the suits to try and automate this one and ignore the lack of quality.
23-0000 Legal Occupations
Although you will probably see the lawyers fight this one to the death, I see a lot of automation coming here. In fact, as we learn just how precisely laws can be stated in the new high-order coding approaches, there could be new standards applied to legal precision as laws are drafted. Forget "I will post legislation on the Internet." The gold standard will be "I will let you log into a game to see just what will be illegal or what will be funded."
25-0000 Education, Training, and Library Occupations
Lots of human touch in evaluating and guiding students. However, a lot of training is also simple drilling and that can be pretty well guided by machines that can distinguish between correct and incorrect performance. On the library side, there will be creative and human-facing work in understanding better approaches to search. Indexing and archiving will be highly automatable.
27-0000 Arts, Design, Entertainment, Sports, and Media Occupations
Totally human-oriented. When a computer learns how to do stand up comedy, we'll be at the Singularity and none of this will matter anyway.
29-0000 Healthcare Practitioners and Technical Occupations
Diagnostic activities will be highly susceptible to automation. Designing treatment regimens may be less so, but again I think we can expect some surprises here.
31-0000 Healthcare Support Occupations
It's a now-familiar divide. Logistics and accounting fall to the machines. Human-facing aspects are hard to replace.
33-0000 Protective Service Occupations
Despite the attractiveness of officers that don't get unduly frightened by people of the wrong race, it's hard to imagine the public allowing robots to carry guns anytime soon. Unless corporations are granted Second Amendment rights along with all the others they've been given.
35-0000 Food Preparation and Service
This one I think is highly dependent on what consumers are willing to accept. I could see a McDonald's of the future that is totally automated with people swiping electronic money cards to complete the transaction. Would people buy it? Who knows? But it's totally possible from a technology standpoint.
37-0000 Building and Grounds Cleaning and Maintenance
Here's where robot mobility and dexterity becomes a big question. It's certainly conceivable that machines could understand how to clean things, repaint them, and plan efficient inspection routes. The question is where this becomes cost effective versus just hiring people. But I don't see any real technical barriers.
39-0000 Personal Care and Service
This is another human-centric set of occupations. So it seems like it would be hard to automate away.
41-0000 Sales and Related Occupations
This one is a mixed bag. Some people like to shop with other people, some prefer Amazon. And there will always be a lot of person-to-person contact in figuring out what sells.
43-0000 Office and Administrative Support Occupations
This is one of those once presumed "safe" families of occupation that are ripe for displacement by soft AI. Classification, application of bodies of rules, and so on are right down the strike zone. There may be a few people around to handle particularly strange circumstances and provide leniency, but a lot of the work could be done by machine.
45-0000 Farming, Fishing, and Forestry Occupations
This is another of those dexterous robot applications, with some quality assessment of the product on top. The hardest jobs to replace in this category are probably things like vegetable pickers. You have to find, reach, and assess ripeness very quickly. Maybe non-visual techniques like looking for trace organic gases coming off the fruit would make the machine more effective. This doesn't seem impossible but the expense of development of these capabilities seems prohibitive, especially while the job is mostly filled by migrant workers.
47-0000 Construction and Extraction Occupations
The same robot dexterity would be required as with the 45 series, but less subjectivity and variability in the end product.
49-0000 Installation, Maintenance, and Repair Occupations
More robot dexterity but these jobs also intersect with the increasing amounts of telemetry generated by different machines. Since devices produce more data while operating, there is more opportunity to extract tell-tale clues to what could be wrong. It also easy to conceive of a future where devices are better designed for machine-assisted repair as well as for modern assembly lines.
51-0000 Production Occupations
Certain kinds of production are already heavily automated. Anything that can be broken done into a time study can be coded into an imperative program. Also, the environment is well-controlled and so provides the most certainty for developers.
53-0000 Transportation and Material Moving Occupations
Driverless cars anyone? In addition, mix in my remarks on 47 and 49 for robot dexterity and on-site situational assessment.
55-0000 Military Specific Occupations
I'd like to say these are human only, but unfortunately no. A lot of military nowadays are crew of armed vehicles like planes and tanks. With dexterous robots, even infantry can be covered. Now there are a lot of human-facing jobs like interpreters, military police, social workers and others.
As you can see, pretty massive swaths of work could be displaced by existing or near-future technology. There are also lots of areas where the technology is not cheap and so may not directly displace workers but rather be used as a threat to extract wage and benefit extractions across the whole economy. And that feature of possible future history is what Parts 3 and 4 of this diary set hinge on. Join in some next time.
