Reply
 
Thread Tools Display Modes
  #201  
Old 08-28-2014, 07:48 AM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
I finally got around to reading Manna. Fascinating story. It's not so much that it's badly written (though I can't argue the point) but that it's written in a style that has been out of fashion for about a century. Very early SF stories used to use a very similar format for describing scientific and social advances that would Change the World. "Professor Formison's Fluxo-logic Capacitor, by altering the variations in the ether, allowed an individual wearing it to ride gravity waves, and thus Nathaniel found himself floating in the air, gazing in wonder at the ground below him, shielded from the harmful effects that would have been incurred by a plunge to his doom by the marvelous abilities of the Fluxo-Logic Capacitor."

It was also commonplace in SF in those days to contrast the marvelous wonderfulness of Society A which had embraced Ringolated Tharianism with the awfulness of Society B which had not done so. Society A would often how towering spires of shining glasslike buildings with the inhabitants frolicking happily about while Society B's inhabitants lived in pathetic hovels made of Terrorfoam and grumbled while eating reconstituted cardboard or something like it. 'Manna" pretty much exactly follows this plan.

Still an interesting read. I think what made the first part so riveting was the sheer plausibility of the initial robotic incursion: workers at fast food restaurants already wear headpieces and operate computer ordering menus, it's so EASY to see all of them wearing headsets and responding to a computer's orders. The software described isn't even an A.I., it's an expert system at best. The step-by-step incursions into human freedom are all very tiny and limited, but you wind up in what I see as not even a worst-case scenario, but one's that nasty enough as it is ... ninety percent of the US population unemployed and unemployable, housed in what are essentially jails.

The section involving the Australia Project is much less interesting because it's practically a rote copy of century-old SF tropes.

Last edited by Evil Captor; 08-28-2014 at 07:49 AM.
  #202  
Old 08-28-2014, 09:47 AM
Whack-a-Mole's Avatar
Whack-a-Mole is offline
Guest
 
Join Date: Apr 2000
Location: Chicago, IL USA
Posts: 20,968
I'm amazed this has not been posted yet. While it may not answer the OP it is relevant I think and certainly interesting:

CGP Grey: Humans Need Not Apply
  #203  
Old 08-28-2014, 03:31 PM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Quote:
Originally Posted by Whack-a-Mole View Post
I'm amazed this has not been posted yet. While it may not answer the OP it is relevant I think and certainly interesting:

CGP Grey: Humans Need Not Apply
An excellent video, thanks for bringing it to our attention. I found it particularly informative to discover there is ALREADY a robot that has solved the hand-eye problem. It's not as fast or as deft as a human being with its robot hands, but it can learn to do anything it sees a human do. It's just a matter of upgrading now.
  #204  
Old 08-28-2014, 03:54 PM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
New thought: could we turn the smarter corporate and independently wealthy minds toward thinking they NEED us as consumers, to maintain their vast wealth? After all, a market for consumer goods that is six billion strong is a LOT more lucrative than a market that is just 350 million strong. Hell, Monsanto alone has a HUGE vested interest in having a lot of hungry mouths to feed, evil bastids though they be.

After all, in the long term consumers may be unnecessary, but in the short term bottom lines are gonna be HURTIN' if too many people lose their ability to buy things. And corporations definitely think in the short term.

I know conservative and libertarians are gonna howl like mad dogs at the very concept of a basic income, much less a generous one, but corporate types tend to be a little bit less ideological in their approach. Whatever works, works, and if giving no-account layabouts a decent living wage (a Goodies for Undesirables program indeed!) is how the corporate bottom line stays healthy and the shareholders stay happy, I think they might suddenly become bastions of generosity.

And the sneakiest fun part of this is, the trend toward control of our democracy by the wealthy might just work to our advantage, and to libertarian and conservatives' disadvantage.
  #205  
Old 11-05-2014, 03:37 PM
SlackerInc's Avatar
SlackerInc is offline
Guest
 
Join Date: Feb 2003
Posts: 11,518
Quote:
Originally Posted by msmith537 View Post
Think of it like this. A couple of hundred years ago, most people worked in agriculture. Since most wealth was generated by the production of crops or livestock, it was a direct function of who owned the most land.

A hundred or so years ago, the Industrial Revolution created a new form of wealth. The ability to make shit and lots of it. People got rich by making lots of stuff or inventing new ways for people to make stuff. It also had the effect of transferring the work force from working on farms to working in factories.

In the past 50 years, it's been about information. As technology and automation has made it easier to make stuff, the focus has shifted to figuring out what to make and how to make it more efficiently. Again, much of the work force has shifted from working in factories to working on the systems that run them.

It stands to reason that in the future, more and more decision making will be handled more cheaply and more efficiently by decision support systems. The company I work for makes software that analyzes data to tell salespeople who they should sell to. A lot of companies have systems that tell them when they should buy. How much longer until those systems are linked and you don't even need salespeople at all?

So when the drudgery of working on farms, factories and IT development centers becomes a thing of the past, what would be the next step? With the ability to produce so much so quickly, society would likely be highly socialized. After all, what does capitalist cutthrough efficiency matter when you can cheaply produce more than most people can ever use?

But what would pass for currency in a society where wealth was no longer based on land ownership, production capacity, or even intellectual property? The only thing left is the ability to amuse and entertain the masses of interchangable carbon blobs sitting in front of their 3D vid walls eating massive piles of cheap food.

You wouldn't teach kids to study math or science or even law or medicine. I would no more want my kids toiling in a data mine than my parents wanted me toiling in a coal mine. You would tell them to spend extra hours at the gym, tanning salon and laundrymat. Your "resume" would be the assorted clips posted on whatever future version of Youtube and Facebook exist.

The world of the future would look a lot like Idiocracy. The difference is that people wouldn't have bred themselves stupid. They would just be stupid because there would be no reason to not be.

Future generations would look at Snookie and The Situation the way we look at Henry Ford or Bill Gates.
This thread is such a pleasure to reread. Everyone posting in it should be proud of their contributions; and I hope it is remembered when future econonic, social, and technological historians take the measure of our age.

The post quoted above, #32, strikes me as a very interesting and unique contribution that got initially lost in the shuffle amidst the embarrassment of riches (if anyone quoted it or responded to it before now, I missed it).

ETA: Captor, you seem to have become more optimistic than you were in 2012.

Last edited by SlackerInc; 11-05-2014 at 03:39 PM.
  #206  
Old 02-01-2015, 02:49 PM
SlackerInc's Avatar
SlackerInc is offline
Guest
 
Join Date: Feb 2003
Posts: 11,518
Here's an interesting twist on this future, that keeps coming at us all the time:

http://www.npr.org/blogs/money/2015/...e-your-machine

In the latter third or so of this podcast, about the Amazon site/service Mechanical Turk*, they pivot from "this allows sites to get things done that require a human touch" to "the people working on this site are helping train computers to hasten their obsolescence". Inevitable, but sadly ironic all the same.

*Which, as an aside, is not a term I'd ever heard before--and the original Turk is a fascinating story as it turns out.

Last edited by SlackerInc; 02-01-2015 at 02:50 PM.
  #207  
Old 02-01-2015, 11:01 PM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Quote:
Originally Posted by SlackerInc View Post
This thread is such a pleasure to reread. Everyone posting in it should be proud of their contributions; and I hope it is remembered when future econonic, social, and technological historians take the measure of our age.

The post quoted above, #32, strikes me as a very interesting and unique contribution that got initially lost in the shuffle amidst the embarrassment of riches (if anyone quoted it or responded to it before now, I missed it).

ETA: Captor, you seem to have become more optimistic than you were in 2012.
I don't know if thinking "the absolute worst may not happen due to corporate greed" is OPTIMISM but it is a cheering thought. On the other hand, I suspect there will be much unnecessary human suffering as I think there are many wealthy conservatives and libertarians who would GLADLY shoot themselves in the foot economically speaking to defend their ideological interests.

Last edited by Evil Captor; 02-01-2015 at 11:02 PM.
  #208  
Old 02-02-2015, 05:57 AM
SlackerInc's Avatar
SlackerInc is offline
Guest
 
Join Date: Feb 2003
Posts: 11,518
There are, but they aren't going to gain control. Even during the time Bush was president and both houses of Congress were in Republican hands, the hardcore libertarian, "drown the government in the bathtub" faction was left on the outside and very frustrated. And with every year, every day even, their most ardent supporters (old white guys) die off and are replaced in the voting pool by 18 year olds who are not down with that vision of government and society.
  #209  
Old 02-05-2015, 09:55 AM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Looks like scientific research is being roboticized. Fascinating, the way new forms of automation and roboticization seem to proceed in an almost random fashion.
  #210  
Old 02-05-2015, 05:18 PM
SlackerInc's Avatar
SlackerInc is offline
Guest
 
Join Date: Feb 2003
Posts: 11,518
That's a big one. It doesn't surprise me, though: I've been counselling my kids for years to go to law school and aim for being trial lawyers or judges, as I think these fields will be legally barred (no pun intended) to robots/AIs for longer than almost any other. The CW is that today's teens should aim for science/engineering/coding type educations for good employment prospects in well paying fields. But I think a lot of that will dry up within a few decades as it gets shifted over to being handled by AIs. Any humans left in the process will be glorified technicians that won't need enough skill to demand high wages (kind of like what happened with photo developing in its latter stages before digital photography took over).
  #211  
Old 04-01-2015, 12:27 PM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Another drop in the bucket: software is automating customer telephone service and legal research jobs in Canada.
  #212  
Old 04-01-2015, 11:03 PM
SlackerInc's Avatar
SlackerInc is offline
Guest
 
Join Date: Feb 2003
Posts: 11,518
Thanks for the update. It strikes me too that lots of jobs have already been "automated" in a more invisible sense. When software tools allow one human being to do what took five or ten previously, those others who are no longer needed have essentially had their job automated even if there are still humans with their job title. We may see people reluctant to just let AIs sit there and make all decisions, but gradually evolve to where there are (to our contemporary standards) shockingly few humans involved--like ten people running a big insurance company or investment bank.
  #213  
Old 04-02-2015, 01:34 AM
Mijin's Avatar
Mijin is offline
Guest
 
Join Date: Feb 2006
Location: Shanghai
Posts: 9,007
Quote:
Originally Posted by SlackerInc View Post
It strikes me too that lots of jobs have already been "automated" in a more invisible sense. When software tools allow one human being to do what took five or ten previously, those others who are no longer needed have essentially had their job automated even if there are still humans with their job title.
Virtually all jobs are already like this, because we've always produced tools to reduce human labor. In that respect, software tools are no different.
  #214  
Old 04-02-2015, 11:25 AM
SlackerInc's Avatar
SlackerInc is offline
Guest
 
Join Date: Feb 2003
Posts: 11,518
Sure, but that trajectory can't be sustained forever. We've managed for a long time to keep finding more work to do beyond fulfilling basic needs, but I think the dropoff in workforce participation shows that we're running out of new needs to create (at least at the rate that requires creating enough software-amplified human jobs to keep everyone employed).

Edited to add: if you go back far enough, you find that this wasn't "always" true either: the rise of agriculture really only benefited a small elite at the expense of the masses.

Last edited by SlackerInc; 04-02-2015 at 11:26 AM.
  #215  
Old 04-02-2015, 12:51 PM
Mijin's Avatar
Mijin is offline
Guest
 
Join Date: Feb 2006
Location: Shanghai
Posts: 9,007
Quote:
Originally Posted by SlackerInc View Post
Sure, but that trajectory can't be sustained forever. We've managed for a long time to keep finding more work to do beyond fulfilling basic needs, but I think the dropoff in workforce participation shows that we're running out of new needs to create (at least at the rate that requires creating enough software-amplified human jobs to keep everyone employed).
I think we're a lot further from that time than some people imagine.
While the workforce participation rate has fallen, modestly, in recent years in the US, it had been growing for a long time before that. There is not yet any pattern of increasing unemployment in the developed world.

But sure, maybe in some future time we'll be post-scarcity or whatever and there won't be jobs (and by and large we won't care).

Quote:
Edited to add: if you go back far enough, you find that this wasn't "always" true either: the rise of agriculture really only benefited a small elite at the expense of the masses.
Even if I agreed with this, I don't see how it's relevant to my use of the word "always".
All I said was that humans have always used tools to reduce the labour needed to do X work. I didn't make any claim about what this meant for the job market.
  #216  
Old 04-02-2015, 07:08 PM
SlackerInc's Avatar
SlackerInc is offline
Guest
 
Join Date: Feb 2003
Posts: 11,518
Earlier in this thread, I had speculated that when humans are no longer needed for conventional productivity, they might still be able to find work as entertainers. But a story I heard on today's Marketplace reminded me that the future is onrushing in that regard as well. It referenced a recent Hollywood Reporter story about how the dead actor Paul Walker is being resurrected, high-tech Weekend at Bernie's style, to keep the megahit Fast and Furious franchise churning out profits:

Quote:
No actor is indispensable. That is the blunt lesson from the fact that Universal Pictures was able to complete its April 3 tentpole, Furious 7, following star Paul Walker's death in a November 2013 car accident about halfway through the shoot.
[...]
Today, it's possible to create an actor entirely from digital composites. The Mill, for example, has completed a 90-second Johnny Walker ad featuring a CG Bruce Lee for agency BBH in China. "We created his entire face in CG and hand-animated that, using shots of the actor for reference," says Shenfield.
[...]
Some movies now even create 3D scans of actors as production begins, so that they can make CG doubles for complicated stunt scenes. It's not known if that occurred on Furious 7, but it was done on Captain America: The Winter Soldier. Explains VFX supervisor Scott Squires: "If there's any inkling that you might need a scan, they scan the actor at the start of production. I've also heard of certain studios having actors scanned just as an archival thing." That way, 20 years from now, filmmakers would still have access to the likeness of that actor — dead or alive.
This kind of matter-of-fact prose really makes me feel like I'm living in the future: it seems like it was not long ago at all that this kind of shit would really freak people out (but then, recently I read about a goat that has spider DNA so it can make spider silk in its milk, and no one seems to be freaking about that either, so I guess I'm just a fuddy-duddy). It reminded me of a movie I saw last year, The Congress, which explored this terrain although kind of abandoned it and took a weird detour just as the premise was catching my interest. But Googling that film, I found this discussion:

Quote:
“Very few people watching that film [Benjamin Button] even knew that [CGI] was going on,” says Darren Hendler, digital effects supervisor at Digital Domain. “We are at the point where we can create a digital version of an actor that is indistinguishable from the real person.”

There are plenty of reasons to welcome the technology. Eventually, it could democratise movies, allowing anybody to make a film using a cast they have created. There are also financial advantages, says Professor Nadia Magnenat Thalmann, who has pioneered research into virtual humans for the past 30 years. “On some films it costs a fortune to hire real actors,” she says. “We’re able to make virtual actors look great – and as soon as we can automate the process, there will be a cost-benefit analysis. If it’s cheaper, second-rank actors will be done more and more by computer.”

According to Hendler, some actors are already embracing the process and having themselves scanned. “If they’re in a movie later on where they need to be younger, they already have that snapshot,” he says. “They are starting to archive their digital selves.” But they’re also wary. Tom Cruise, who was scanned for his role in Oblivion, had his data hand-delivered to his house and all other copies destroyed. “The more experienced actors will start to have more control over their contracts with the studios; more say over how their final digital likeness looks and how it is used,” says Hendler.

But if Folman’s vision becomes a reality, films might even be cast using digital versions of actors who have long since died. It is already possible to create fairly convincing virtual versions of actors who were never scanned, using old footage and performance doubles. And, a CG version of Paul Walker has been created to complete Fast and Furious 7, after the actor died part-way through filming.

There are also developments that could make real actors, even in their digital forms, completely redundant. “The big trend is to make virtual humans conscious of their environment and intelligent,” says Magnenat Thalmann. If successful, film-makers will one day be able to create virtual actors who respond to their direction autonomously.
That was smart of Tom Cruise, but the finger will not stay in the dike forever. I think the idea is revoltingly ghoulish, personally. Just make brand new virtual actors instead, that are perfectly engineered to be super hot, eternally youthful, but with just the right amount of "imperfection". That's kind of messed up, too, but at least it's not taking an actual person's image from beyond the grave and making them dance like a marionette in a way they never agreed to.
  #217  
Old 04-12-2015, 08:36 PM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Japan to open hotel entirely staffed by robots in 2015.

Also, more inroads on customer service and medical work. Also legal, but I think that part was covered in a previous post.
  #218  
Old 04-14-2015, 09:58 AM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Robot chef can make 2000 different dishes. Restauranteurs will probably view this with interest.
  #219  
Old 04-14-2015, 11:56 AM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
LaGuardia Airport automates cashiers.
  #220  
Old 04-14-2015, 10:27 PM
Mijin's Avatar
Mijin is offline
Guest
 
Join Date: Feb 2006
Location: Shanghai
Posts: 9,007
Quote:
Originally Posted by Evil Captor View Post
Robot chef can make 2000 different dishes. Restauranteurs will probably view this with interest.
Yeah, I'm skeptical about that one. Firstly the 2,000 dishes is a goal; it's not clear how many it can do right now. In the Economist's article, the implication was right now it can just do one dish, after you lay out the (chopped) ingredients for it.
The main reason I'm skeptical is because I think using standard kitchen equipment and motion capture is inherently bug-prone.

Not that I think an automated cook is some kind of fantasy idea; it's something that would be incredibly useful / profitable and the technology does appear to be in place. I just suspect the first robust system will either use custom apparatus to cook with, or use sensors, cameras etc to interact in a standard kitchen.
  #221  
Old 04-17-2015, 04:16 PM
SlackerInc's Avatar
SlackerInc is offline
Guest
 
Join Date: Feb 2003
Posts: 11,518
The first iteration (or even the first few) are always going to be problematic. But this is an area I hadn't even thought of, yet seems so obvious on reflection. I would expect that in our lifetime, only very fancy or boutique-y restaurants (along with maybe a few struggling and straggling greasy spoons) will retain human kitchen staff (servers might hold out longer, even though their job is more easily automated in some ways, just because of customer preferences).
  #222  
Old 04-17-2015, 04:41 PM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Quote:
Originally Posted by Mijin View Post
Yeah, I'm skeptical about that one. Firstly the 2,000 dishes is a goal; it's not clear how many it can do right now. In the Economist's article, the implication was right now it can just do one dish, after you lay out the (chopped) ingredients for it.
The main reason I'm skeptical is because I think using standard kitchen equipment and motion capture is inherently bug-prone.

Not that I think an automated cook is some kind of fantasy idea; it's something that would be incredibly useful / profitable and the technology does appear to be in place. I just suspect the first robust system will either use custom apparatus to cook with, or use sensors, cameras etc to interact in a standard kitchen.
I agree, it's a just a bare beginning, hardly a well-developed technology, and I agree that motion capture has a lot of inherent limitations that will cause problems. And I suspect that the way robotic kitchens will evolve is as a whole series of devices that make cooking easier and easier, reducing kitchen staff until a critical mass is reached and the staff required becomes less than one.
  #223  
Old 05-04-2015, 08:58 AM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
A robotic factory that makes robots is capable of running for 30 days at a time without human intervention, it's part of a trend toward lights-out manufacturing, and has been in existence since 2001. The writing is on the wall for manufacturing work has been around for a while: old news, but people have been wondering who will build the robots. The answer is clear: other robots.
  #224  
Old 05-04-2015, 12:09 PM
Ruken is offline
Charter Member
 
Join Date: May 2003
Location: DC
Posts: 7,336
Quote:
Originally Posted by Evil Captor View Post
Looks like scientific research is being roboticized. Fascinating, the way new forms of automation and roboticization seem to proceed in an almost random fashion.
This reminded me that Marty Burke published a paper on automated chemical synthesis a month or two back in Science. I will try to return with links and commentary when I'm not on the road.
  #225  
Old 05-04-2015, 02:41 PM
SlackerInc's Avatar
SlackerInc is offline
Guest
 
Join Date: Feb 2003
Posts: 11,518
Quote:
Originally Posted by Evil Captor View Post
A robotic factory that makes robots is capable of running for 30 days at a time without human intervention, it's part of a trend toward lights-out manufacturing, and has been in existence since 2001. The writing is on the wall for manufacturing work has been around for a while: old news, but people have been wondering who will build the robots. The answer is clear: other robots.
There's something creepy about "lights out" manufacturing, though it makes perfect sense logically.
  #226  
Old 05-05-2015, 06:24 AM
Ruken is offline
Charter Member
 
Join Date: May 2003
Location: DC
Posts: 7,336
Quote:
Originally Posted by Ruken View Post
This reminded me that Marty Burke published a paper on automated chemical synthesis a month or two back in Science. I will try to return with links and commentary when I'm not on the road.
Still on the road, but breakfast isn't open yet. Found it:
DOI: 10.1126/science.aaa5414
Easy-read: http://pipeline.corante.com/archives..._synthesis.php

I expect this will have affects similar to the automated synthesis of peptides and nucleic acids, i.e. more, cooler science to be done.
  #227  
Old 05-05-2015, 08:37 AM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Quote:
Originally Posted by Ruken View Post
Still on the road, but breakfast isn't open yet. Found it:
DOI: 10.1126/science.aaa5414
Easy-read: http://pipeline.corante.com/archives..._synthesis.php

I expect this will have affects similar to the automated synthesis of peptides and nucleic acids, i.e. more, cooler science to be done.
I'm not sure what this has to do with robotic and automation taking away jobs, though I saw hints when someone n the comments said it might pave the way for cheap home molecular printers. It sounds like it might put a few organic chemists out of work, but more likely just force them to move on to new fields: it seemed to work only with certain kinds of chemical bonds.
  #228  
Old 05-05-2015, 09:36 AM
Ruken is offline
Charter Member
 
Join Date: May 2003
Location: DC
Posts: 7,336
Quote:
Originally Posted by Evil Captor View Post
I'm not sure what this has to do with robotic and automation taking away jobs, though I saw hints when someone n the comments said it might pave the way for cheap home molecular printers. It sounds like it might put a few organic chemists out of work, but more likely just force them to move on to new fields: it seemed to work only with certain kinds of chemical bonds.
It reduces a major component of drug development that takes years of work and a PhD in chemistry to a few minutes of button-pushing and an afternoon of waiting.

Not in all cases, but in enough to disrupt the industry.
  #229  
Old 05-05-2015, 10:28 AM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Quote:
Originally Posted by Ruken View Post
It reduces a major component of drug development that takes years of work and a PhD in chemistry to a few minutes of button-pushing and an afternoon of waiting.

Not in all cases, but in enough to disrupt the industry.
Ah, gotcha. That would explain many of the comments after the blog post.
  #230  
Old 05-10-2015, 08:29 AM
SlackerInc's Avatar
SlackerInc is offline
Guest
 
Join Date: Feb 2003
Posts: 11,518
This episode of the Planet Money podcast is germane to the discussion, especially 45 seconds of it near the end: from 18:00-18:30, and 19:10-19:25.
  #231  
Old 05-10-2015, 09:58 AM
Mr. Nylock is offline
Guest
 
Join Date: Jan 2014
Location: Bed
Posts: 3,093
Quote:
Originally Posted by Hellestal View Post
This is contrary to centuries of history, and even the most introductory economics.

Even when some workers do lose out -- which can absolutely happen, yes, if we're not on a super-long-term time frame -- the owners of the robots are not now, nor have they ever been, the only winners. The remaining workers benefit loads, as does the rest of society. We've had two hundred years of evidence about this since the beginning of the Industrial Revolution. It is helpful to learn a little bit about these events and economic processes that led to our modern technological world in order to comment sensibly about it.
I find this argument very weak. The future of automation will be very different from the past, and encroach more and more on white collar work.

I find this analysis of history as tremendously flawed in the situation. In the beginning of the industrial revolution, it was horrible for the workers. It was not as if technology was created and the benefits just trickled down to everyone - they had to be fought for. The workers fought and won - but part of the reason they won is because they were needed; they were a necessary part of the production process. If workers are no longer part of the production process, and therefore not needed, then what leverage do they have?

Last edited by Mr. Nylock; 05-10-2015 at 10:01 AM.
  #232  
Old 05-10-2015, 10:19 AM
Hellestal is offline
Charter Member
 
Join Date: Aug 2002
Location: Storyland
Posts: 2,037
Quote:
Originally Posted by Mr. Nylock View Post
I find this argument very weak. The future of automation will be very different from the past, and encroach more and more on white collar work.

I find this analysis of history as tremendously flawed in the situation. In the beginning of the industrial revolution, it was horrible for the workers. It was not as if technology was created and the benefits just trickled down to everyone - they had to be fought for. The workers fought and won - but part of the reason they won is because they were needed; they were a necessary part of the production process. If workers are no longer part of the production process, and therefore not needed, then what leverage do they have?
You're responding to a post from January 2012.

There is a more recent, and much higher quality thread, from just last month where my thoughts on this topic are more fully fleshed out. If you'd like to discuss this, you'd be better off reading those posts to get more context and then bumping that thread with any questions or criticisms you have.

One thing I'd briefly point out, which I might not have mentioned there, is that it's an extremely difficult argument to make that early automation was worse for workers than what had come before. Enclosure preceded the Industrial Revolution proper. So the actual question is: Did the early Industrial Revolution literally make things worse, or did it consolidate already existing poverty into focal points that were no worse than before, but more easily observed and commented upon? (I don't have an answer to that, by the way. I want to emphasize the problem with casual historical assertions. It's not enough to point at bad things and say they're bad, when those conditions might plausibly have been an improvement on what they were doing before.)

Last edited by Hellestal; 05-10-2015 at 10:20 AM.
  #233  
Old 05-10-2015, 02:39 PM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Another drop in the bucket: an autonomous driving truck is cleared for use on US highways.
  #234  
Old 05-10-2015, 02:45 PM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Quote:
Originally Posted by Mr. Nylock View Post
I find this argument very weak. The future of automation will be very different from the past, and encroach more and more on white collar work.

I find this analysis of history as tremendously flawed in the situation. In the beginning of the industrial revolution, it was horrible for the workers. It was not as if technology was created and the benefits just trickled down to everyone - they had to be fought for. The workers fought and won - but part of the reason they won is because they were needed; they were a necessary part of the production process. If workers are no longer part of the production process, and therefore not needed, then what leverage do they have?
Well that's my viewpoint too ... I'm sure that, long term, automating jobs out of existence will be good for the human race ... at least, those who survive the process. But looking at the trends today: huge wealth inequality sparking a huge social gap between the one percent and the peasantry, libertarianism and conservatism increasingly popular among the one percent, Citizens United making our government basically a puppet of the One Percent ... it looks BAD for our children, possibly our children's children.
  #235  
Old 05-10-2015, 03:03 PM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Quote:
Originally Posted by Hellestal View Post
You're responding to a post from January 2012.

There is a more recent, and much higher quality thread, from just last month where my thoughts on this topic are more fully fleshed out. If you'd like to discuss this, you'd be better off reading those posts to get more context and then bumping that thread with any questions or criticisms you have.

One thing I'd briefly point out, which I might not have mentioned there, is that it's an extremely difficult argument to make that early automation was worse for workers than what had come before. Enclosure preceded the Industrial Revolution proper. So the actual question is: Did the early Industrial Revolution literally make things worse, or did it consolidate already existing poverty into focal points that were no worse than before, but more easily observed and commented upon? (I don't have an answer to that, by the way. I want to emphasize the problem with casual historical assertions. It's not enough to point at bad things and say they're bad, when those conditions might plausibly have been an improvement on what they were doing before.)

Here's a piece
the addresses the living and working conditions of people in Britain during the Industrial Revolution. It's even worse than I thought, and I thought it really sucked!

Quote:
The Registrar General reported in 1841 that the average life expectancy in rural areas of England was 45 years of age but was only 37 in London and an alarming 26 in Liverpool (Haley).
Life expectancy in industrial Liverpool was very nearly half what it was in rural England. I'd say that tells us a lot about the quality of life of the rural poor vs. the urban poor.
  #236  
Old 05-10-2015, 09:24 PM
Hellestal is offline
Charter Member
 
Join Date: Aug 2002
Location: Storyland
Posts: 2,037
Quote:
Originally Posted by Evil Captor View Post
Life expectancy in industrial Liverpool was very nearly half what it was in rural England. I'd say that tells us a lot about the quality of life of the rural poor vs. the urban poor.
That is a good and important fact.

But it doesn't address the underlying issue I was referring to. Enclosure preceded the Industrial Revolution. Enclosure forced masses of people out of the countryside and into the relatively less healthful cities, and this would have been true even without a subsequent Industrial Revolution. There was always going to be a mass of sickly people living in urban areas after enclosure and before modern sanitation. The question is whether the subsequent factory system contributed on average to those urban problems, or provided some slight amelioration to what their existence otherwise would have been.

That issue can't be resolved with simple life expectancy comparisons. It is at least an order of magnitude more difficult because it involves imagining what their lives would have been like without the Revolution ever happening.
  #237  
Old 05-11-2015, 03:51 AM
SlackerInc's Avatar
SlackerInc is offline
Guest
 
Join Date: Feb 2003
Posts: 11,518
Quote:
Originally Posted by Hellestal View Post
There is a more recent, and much higher quality thread,
Much higher? Well, first of all, that's just rude. I'm partial to this one, of course, but it seems to me that they are both of good quality. (The best posts in both threads are those by Lemur866, IMO.)
  #238  
Old 05-11-2015, 03:55 AM
SlackerInc's Avatar
SlackerInc is offline
Guest
 
Join Date: Feb 2003
Posts: 11,518
Quote:
Originally Posted by Evil Captor View Post
Another drop in the bucket: an autonomous driving truck is cleared for use on US highways.
That's fascinating. I'm surprised they are this far along--but glad to see this, as it could really be a boon for safety.

If you read through the article, though, it still requires having a driver. Which hints at one way automation could go: rather than lay everyone off and put them on the dole, people might keep "working" alongside the automation, so that the future looks a little like The Jetsons, where George's job involved pushing the occasional button.
  #239  
Old 05-11-2015, 07:12 AM
Ruken is offline
Charter Member
 
Join Date: May 2003
Location: DC
Posts: 7,336
Quote:
Originally Posted by SlackerInc View Post
Which hints at one way automation could go: rather than lay everyone off and put them on the dole, people might keep "working" alongside the automation.
That's why automation creates jobs. It always has and does so today.
  #240  
Old 05-11-2015, 07:25 AM
Atomic Alex's Avatar
Atomic Alex is offline
Guest
 
Join Date: Sep 2009
Posts: 2,843
I recently read the 'Yukikaze' novels by Chōhei Kanbayashi which had a rather interesting depiction of the interrelationship between advanced AI's and humanity in a military context. I thought it had a rather neat explanation of why humans were still a necessary component in the system:

SPOILER:
During the climatic battle the airbase is under both physical and electronic attack, the AI systems aren't sure if what they are receiving from their sensory equipment is real or the result of sustained hacking attacks, so they request humans to check the reports out confirm or deny what is actually occurring, as humans are biological and can't be hacked in the same way


In fact for most of the war the alien enemy (the JAM) don't seem to perceive humans at all and humans have never seen a JAM only their machines (its unclear if the JAM *are* their machines or not) so you have humans building machines to fight machines and the machines creating humans to fight humans.

They are actually really good books with quite a few ideas I haven't come across elsewhere. The anime has its moments but it isn't nearly as good.

*and apparently they're making a live-action movie starring Tom Cruise! They made a great adaptation of 'All You Need Is Kill' as 'Edge of Tomorrow', so fingers crossed.

Last edited by Atomic Alex; 05-11-2015 at 07:28 AM.
  #241  
Old 05-11-2015, 09:24 AM
Hellestal is offline
Charter Member
 
Join Date: Aug 2002
Location: Storyland
Posts: 2,037
Quote:
Originally Posted by SlackerInc View Post
Much higher? Well, first of all, that's just rude. I'm partial to this one, of course, but it seems to me that they are both of good quality.
Look at the post participation statistics. As of this moment, the top standings are:
Code:
SlackerInc	65
Evil Captor	49
Mijin	 	15
Lemur866	12
Quite a gap between the number two and number three positions, yes?

There's nothing inherently wrong about having a place to keep noting the progression of technology. Bump when you see something interesting. But not everybody is interested in reading a blog. The only reason I even opened this thread was that I saw the most recent post was from someone I knew for a fact joined up more recently, on double-checking I see this poster arrived two years after the thread was started. If the name had been another one of the top-two regulars, I wouldn't have bothered.

Endless bumping doesn't make a coherent discussion. It's interesting for some but not for everybody. This is the basis of the opinion I offered on thread quality. You are free to believe that offering opinions in GD is "rude" but you can't deny that there is an objective difference in the nature of the two threads, and that some people will prefer one style of participation to the other.
Quote:
Originally Posted by SlackerInc View Post
(The best posts in both threads are those by Lemur866, IMO.)
Lemur was participating in this thread in April of 2012. Three years ago, in other words. Then a gap of more than two years with no participation. There was another brief spurt of participation in August of last year. If Lemur were the second name on that participation list, I have no doubt that this thread would be excellent. But the reality is that the top two posters have almost half of total posts. Number two on the list has more than triple the posts of number three.

That's absolutely fine, for people who like that sort of thing. If it were my thread, I'm sure I wouldn't mind at all. But not everybody else will feel the same way.
  #242  
Old 05-11-2015, 01:44 PM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Here's another low-quality post, an excerpt from a book about the threat of automation to the job sector from Salon.com. The excerpt deals with automation in the retail sector, especially fast food.
  #243  
Old 05-11-2015, 02:29 PM
Lemur866's Avatar
Lemur866 is offline
Charter Member
 
Join Date: Jul 2000
Location: The Middle of Puget Sound
Posts: 22,394
Quote:
Originally Posted by Hellestal View Post
If Lemur were the second name on that participation list, I have no doubt that this thread would be excellent. But the reality is that the top two posters have almost half of total posts. Number two on the list has more than triple the posts of number three.

That's absolutely fine, for people who like that sort of thing. If it were my thread, I'm sure I wouldn't mind at all. But not everybody else will feel the same way.
Jesus, if only there were some way to automate my participation in this thread, to keep the complaints about my absence down to a dull roar....

It seems to me we've had several interlocking threads on this topic, and I pretty much keep saying the same sorts of things.

One thing I would emphasize is that automation is going to be destroying lots and lots and lots of white collar professional jobs. We keep talking about automation destroying blue collar jobs, and it has done that and will continue to do that. But simply because the past elimination of blue collar jobs has been so thorough and so relentless the existing jobs tend to be ones that have been, so far, resistant to that sort of loss.

So we can build an automated assembly line where the welders and bolt-turners are replaced by machines. But a home health care worker is a lot harder. A robot that can change a bedpan isn't in the cards any time soon. You need a radically different sort of technology that doesn't replace human health aides with robot health aides, but fills the same need in a different way.

And that's what is happening with white collar jobs. Lots of the work that used to be done by law clerks flipping through books and copying down citations is now done by a 10 second database query. But that doesn't eliminate jobs for lawyers or paralegals if the demand for legal services grows along with productivity. That's the key part--tools that greatly increase the productivity of white collar workers dramatically increase the potential upside for the industry, but are eventually going to mean fewer and fewer workers in that industry without phenomenal growth in that industry.

It seems to me that here in 2015 we have a huge store of underutilized capital, but not much demand for new production. Companies are sitting on huge piles of cash, but don't have many good ideas on how to put that money to work.
  #244  
Old 05-11-2015, 03:02 PM
Lemur866's Avatar
Lemur866 is offline
Charter Member
 
Join Date: Jul 2000
Location: The Middle of Puget Sound
Posts: 22,394
And one more thing that I've also said before.

A future of billions of gleaming automated vending machines that can produce any good or service imaginable if you only put in a quarter, with the starving unwashed masses walking by wishing they had a quarter to put in, doesn't make any sense.

There won't be any such machines in that economic system, because why would the capitalist overlords build them, knowing the masses don't even have a quarter to put into them? Capitalists own businesses to make money. One way to do that is to have the business produce goods and services that people purchase in exchange for money. If no one exchanges money for those goods and services, then there's no way to make money by producing them, and therefore no capitalist would ever do such a thing.

OK, so there are no robot factories tirelessly making the goods and services the masses need for survival, because the masses can't afford to pay for those goods and services. So now the masses can have jobs after all, because those factories that would put them out of work don't exist! Of course there's an equilibrium here, but the point is that you can't get rich being a capitalist unless you produce goods and services people are willing and able to pay for. But some goods and services might be so cheap to provide that you can do that for free, just with the understanding that you're not making much money doing it. And so we have Craigslist that makes a very modest profit by automated distribution of classified ads, destroying thousands of jobs. But if Craigslist tried to capture the revenue that all the paper newspapers used to get for serving classified ads, they'd fail spectacularly. In 1970 if you could somehow siphon off all the revenue that all the newspapers in America used to generate through classified ads, you'd be filthy rich. In 2015 that isn't possible. It's possible to provide those services for free and make a tiny profit, it's impossible to get filthy rich doing it.

And this is the future of whole categories of services that are currently generating hundreds of billions of dollars for the owners of various companies. Job categories that are likely to be eliminated by automation aren't just "burger flipper" and "truck driver", they're also "CEO of FaceGoogle Sony-Warner".
  #245  
Old 05-11-2015, 03:56 PM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Meanwhile, in Germany, things are looking catastrophic according to a recent study. Interestingly, and in direct contradiction to the post above, the study above, the prediction is that it's the low-wage, lower-skilled workers who will lose out first and foremost. Its predictions are catastrophic:

Quote:
The results of the study paint an almost doomsday-esque scenario for Germany.

Almost two thirds of its workforce will be unemployed. Of the 30.9 million people currently in full or part-time employment in Germany, 18 million will be made redundant by improved technology, the report claims.
One bright spot: the report does not set forth any kind of time frame for the job losses. (The article only mentions "coming decades.") To my mind the report is kind of meaningless without a time frame: they're just saying that at some point in the future 18 million jobs will be lost. The severity of the robot job holocaust is going to be a matter of time as well as numbers. And of course, inability to generate new jobs that won't also be more suitable for robots. This is the thing the optimists keep missing. Robots and computers aren't the inflexible, purpose-built machines of the past. They can be programmed to perform new tasks. Their skills at performing those tasks can be upgraded quickly and easily via software.

This new Industrial Revolution is a whole different animal than its predecessors.

Last edited by Evil Captor; 05-11-2015 at 03:57 PM.
  #246  
Old 05-11-2015, 04:24 PM
Evil Captor is offline
Guest
 
Join Date: Apr 2002
Location: Lair
Posts: 20,890
Quote:
Originally Posted by Lemur866 View Post
And one more thing that I've also said before.

A future of billions of gleaming automated vending machines that can produce any good or service imaginable if you only put in a quarter, with the starving unwashed masses walking by wishing they had a quarter to put in, doesn't make any sense.

There won't be any such machines in that economic system, because why would the capitalist overlords build them, knowing the masses don't even have a quarter to put into them? Capitalists own businesses to make money. One way to do that is to have the business produce goods and services that people purchase in exchange for money. If no one exchanges money for those goods and services, then there's no way to make money by producing them, and therefore no capitalist would ever do such a thing.
You are assuming intelligence not in evidence. Do you remember the Crash of 2007? That was capitalist bankers driving the economy into a ditch, laughing all the way. I am not optimistic that capitalists in any other sector of the economy are any smarter or more moral.
  #247  
Old 05-11-2015, 11:05 PM
SlackerInc's Avatar
SlackerInc is offline
Guest
 
Join Date: Feb 2003
Posts: 11,518
Disposable Hero, that book sounds cool. I would note that DARPA has done a lot of research on autonomous military robots, and I certainly hope that for some time to come they are going to keep double-checking with human overseers before going into "KILL! DESTROY!" mode. But a point that was made on a podcast I listened to about it was that this may not be possible if the speed of combat gets to a point that human operators/overseers are just too slow to keep up. I guess we can hope at least that they can press a button to stop further carnage if something is going terribly awry.

The storyline you describe also reminds me of one in the (IMO very underrated) show Stargate Universe:

SPOILER:
They encounter swarms of AI drones who have been programmed to attack and destroy any species other than the one that created them. Said species went extinct long before, however, so the drones are essentially bent on annihilating all life, everywhere.


Quote:
Originally Posted by Ruken View Post
That's why automation creates jobs. It always has and does so today.
If you listen to the segment of the Planet Money podcast I linked to, they point out that it's not so simple. A couple hundred years later, we can say "oh, those silly Luddites--we all ended up better off due to the automation they opposed". But the experts they talked to noted that it was actually rational self-interest for the Luddites to attempt to do what they did, because the automation did actually kill jobs and depress wages for a half century at least. That was the entire rest of their working lifetimes.

To us now, a fifty year rough patch in the 19th century is some historical bump in the road. But if we hit another such patch in the 21st century, it will be cold comfort to think "well, things will sort out by the time my great-grandkids hit the workforce". I mean, as Keynes wryly observed, "in the long run, we're all dead".

I hadn't noticed when I linked that podcast upthread that it had a transcript. Here are the relevant portions:

Quote:
GOLDSTEIN: I mean, it's, you know, it's tempting to shout back across 200 years of history to the Luddites and just say, like, trust me, trust me, things are going get so much better. You Luddites, you are on the wrong side of history.

KESTENBAUM: The truth though is that for the Luddites - the weavers, the guys with the big scissors - things did not get better. They didn't get better in their lifetimes. Things did not even get better for their kids.

GOLDSTEIN: When you step back, when you look at what happened in England as a whole around this time, it is shocking. For 50 years, in this moment when England is building really what is the first modern high-tech global economy on the planet, they're creating all these industrial fortunes, but average wages for workers don't really budge.

KESTENBAUM: Few people make a lot more. It's a good time to be building factories or repairing machines, but lots of people make less.

ALLEN: The winners won and the losers lost, and that was all there was to it.

KESTENBAUM: Were the Luddites right then?

ALLEN: Well, it was certainly I think in their interest to wreck machines. They were acting rationally, and I think to say that they were irrational and opponents of progress and deluded is a big mistake.

KESTENBAUM: We are living now in the middle of a second machine age. It's computers and software this time, not weaving machines. But some of the same things are happening. You hear about the rise of the 1 percent, about how income for ordinary people is stagnant. Part of that is caused by technology.

GOLDSTEIN: And the traditional economic response to this is these problems are temporary. Technology makes everybody better off in the long run, but one of the things the Luddites have to teach us is the long run can be really, really long.
[...]
KESTENBAUM: We'll have more stories about machines taking jobs. There are some very smart people out there arguing that this second machine age is going to be very different than what the Luddites went through, that when computers and robots take jobs, those jobs will not come back. There will just be fewer jobs, period.
I would argue that the steady drop in the workforce participation rate is a key indicator to watch. I think the unemployment rate is likely to bounce around a reasonable number, but I would not expect that participation rate to go back up any time soon, if ever.

Quote:
Originally Posted by Lemur866 View Post
Jesus, if only there were some way to automate my participation in this thread, to keep the complaints about my absence down to a dull roar....
Heh, well played.

I agree with you that certain white collar fields are likely to be particularly impacted in the short and medium term. While I take the general optimistic view, I don't think it's wise to spend time and money focusing on learning certain skills and trades that don't seem to have a lot of staying power. I'm thinking of things like X-ray tech and, as you say, paralegal (I am encouraging my son to be a trial lawyer and maybe a judge, as I think it will be a long time if ever before society allows either of these jobs to be automated completely). I also wonder about a lot of the jobs in engineering and coding that kids are being pushed toward.
  #248  
Old 05-16-2015, 02:03 PM
Ruken is offline
Charter Member
 
Join Date: May 2003
Location: DC
Posts: 7,336
http://www.amazon.com/Rise-Robots-Te.../dp/0465059996
Martin Ford's Rise of the Robots: Technology and the Threat of a Jobless Future came out this month. It is reviewed in the WSJ: http://www.wsj.com/articles/soon-the...too-1431723233
  #249  
Old 05-16-2015, 08:33 PM
Atomic Alex's Avatar
Atomic Alex is offline
Guest
 
Join Date: Sep 2009
Posts: 2,843
Quote:
Originally Posted by SlackerInc View Post
Disposable Hero, that book sounds cool. I would note that DARPA has done a lot of research on autonomous military robots, and I certainly hope that for some time to come they are going to keep double-checking with human overseers before going into "KILL! DESTROY!" mode. But a point that was made on a podcast I listened to about it was that this may not be possible if the speed of combat gets to a point that human operators/overseers are just too slow to keep up. I guess we can hope at least that they can press a button to stop further carnage if something is going terribly awry.

The storyline you describe also reminds me of one in the (IMO very underrated) show Stargate Universe:

SPOILER:
They encounter swarms of AI drones who have been programmed to attack and destroy any species other than the one that created them. Said species went extinct long before, however, so the drones are essentially bent on annihilating all life, everywhere.
I really do highly recommend it, from the cover and synopsis it seems like its just another 'alien invasion/fast-jets and explosions everywhere' piece of fluff (though I'd be perfectly OK with that) but its actually quite thought-provoking on several different levels.

Never watched Stargate Universe but that does sound plausible!

On a different note although I've heard about humans being removed from future combat aircraft for some time I never really understood it on a visceral level until I watched some youtube videos of model helicopters and drones being flown in a way that would turn a human into pulp and would be extremely difficult to combat as an opponent.

Quote:
Originally Posted by SlackerInc View Post
I'm thinking of things like X-ray tech and, as you say, paralegal (I am encouraging my son to be a trial lawyer and maybe a judge, as I think it will be a long time if ever before society allows either of these jobs to be automated completely). I also wonder about a lot of the jobs in engineering and coding that kids are being pushed toward.
I've read several sci-fi stories where humans are indeed removed from the legal process on the premise that a sufficiently advanced AI would be a truly neutral and impartial upholder of the law, making decisions based on the facts of the case alone without those awkward human prejudices and emotions getting in the way.

But we're a long way from that yet.

Last edited by Atomic Alex; 05-16-2015 at 08:33 PM.
  #250  
Old 05-16-2015, 08:47 PM
Mr. Nylock is offline
Guest
 
Join Date: Jan 2014
Location: Bed
Posts: 3,093
Quote:
Originally Posted by SlackerInc View Post
I agree with you that certain white collar fields are likely to be particularly impacted in the short and medium term. While I take the general optimistic view, I don't think it's wise to spend time and money focusing on learning certain skills and trades that don't seem to have a lot of staying power. I'm thinking of things like X-ray tech and, as you say, paralegal (I am encouraging my son to be a trial lawyer and maybe a judge, as I think it will be a long time if ever before society allows either of these jobs to be automated completely). I also wonder about a lot of the jobs in engineering and coding that kids are being pushed toward.
I'm studying accounting now. I'm taking the upper level classes and I'll be ready to sit for the CPA exam (the gold standard in accounting) soon. People keep telling me that they can't automate what a CPA does - I'm seriously finding it hard to believe. I think I've got 10, maybe 15 years to make some money at it before the wages start getting severely depressed by automation.

Last edited by Mr. Nylock; 05-16-2015 at 08:48 PM.
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 03:58 AM.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.

Send questions for Cecil Adams to: cecil@straightdope.com

Send comments about this website to: webmaster@straightdope.com

Terms of Use / Privacy Policy

Advertise on the Straight Dope!
(Your direct line to thousands of the smartest, hippest people on the planet, plus a few total dipsticks.)

Copyright © 2018 STM Reader, LLC.

 
Copyright © 2017