DeepMind AI Exterminates Space Invaders, Pac-Man

Google's DeepMind project has developed an artifical intelligence algorithm that can pretty much teach itself from scratch. For now, its biggest accomplishment is mastery of 49 old Atari arcade games, including Space Invaders and Pac-Man. As machine learning continues to advance, however, its potential for doing great good may be tempered by the risk that it could do great harm.

Researchers at Google's DeepMind subsidiary in England have developed an artificial agent they call a "deep Q-network" that learned to play 49 classic Atari 2600 arcade games by just diving in.

The DQN algorithm performed at more than 75 percent of the level of a professional player in more than half the games, and came up with far-sighted strategies that let it achieve the maximum attainable scores in certain games, such as Breakout.

IT outperformed previous machine learning methods in 43 of the games.

DQN was able to work straight out of the box across all the games, given only the raw screen pixels, the set of available actions, and game score.
"There is nothing bespoke about the system that's tailored to Atari games, specifically; they're just a very handy and useful testing ground," said Demis Hassabis, VP of engineering at Google DeepMind.

"The idea is for future versions of this system to be able to generalize to tackle any sequential decision-making problem even when the input data is very high-dimensional," he told TechNewsWorld.
How DQN Works

Several features enabled the power of deep neural networks to be combined scalably with reinforcement learning for the first time in DQN.

Reinforcement learning, or RL, is an area of machine learning that doesn't present correct input/output pairs or explicitly correct suboptimal actions, unlike standard supervised learning. It focuses on finding a balance between exploration of uncharted territory and exploitation of current knowledge. In other words, it's a sink-or-swim approach.

A feature called "experience replay" played a key role in combining deep neural networks, or DNN, with RL. DQN was trained on samples drawn from a pool of stored episodes during the learning phase, similar to what's physically realized in the brain through flashbacks or dreams. That was critical to DQN's success -- disabling it led to a severe deterioration in performance.
Rise of the Machines?

The fear that machines could take over our world has been expressed not only in sci-fi stories such as The Terminator and Battlestar Galactica, but also by visionaries such as Stephen Hawking, Elon Musk and Bill Gates -- all of whom have expressed concerns about untrammeled research into artificial intelligence.

Ironically, Musk had invested in DeepMind, which was purchased by Google in January of 2014, precisely because of his concerns over the dangers of AI.

"There should be ways in which there's control by humans," Frost & Sullivan analyst Mukul Krishna told TechNewsWorld in an earlier discussion on this issue. "The human world is extremely nuanced. There are too many abstractions that are not logical."
Where AI Will Come in Handy

In certain scenarios, such as in medicine and in the financial markets, AI might provide an advantage, said Dan Kara, practice director for robotics at ABI Research.

For example, diagnostics could be supported by opinions and outcomes drawn from very large databases of historical cases that are analyzed, connected to others worldwide, and continually updated.

The stock markets already use AI tools and techniques for predictive modeling and for making complex trades, but they "will be dwarfed by future systems" that will be able to learn as they go, Kara told TechNewsWorld.

Those programs will be able to learn from other trading systems, as well as draw conclusions from seemingly unrelated events occurring throughout the world, Kara suggested.
Miles to Go Before Humanity Sleeps

Forget the Terminator scenario for now, said Jim McGregor, principal analyst at Tirias Research. It "would require advanced robotics combined with a machine capable of supporting a very high number of unimaginably complex algorithms and unlimited memory, especially for real-time interactions."

In the near term, research on AI will lead to a re-evaluation of how we architect and program computing solutions "that includes everything from the silicon up through the networks," he told TechNewsWorld. In the long run, it will lead to advancements in "everything from medicine to space travel."

The Apple Watch Mystery Won't Be Solved March 9

The fog shrouding Apple's first major new product entry since the iPad -- the Apple Watch -- won't be lifted on March 9 when Apple is widely expected to tell us more about its forthcoming Apple Watch release.

Apple's invite to its media event is characteristically coy, with its vague "Spring Forward," tease, which likely refers to the shift in daylight savings time -- and well, time itself -- but the event possibly could include a new MacBook Air, iPad Pro, or an Apple TV/Center-of-Your-Universe-Hub announcement, as well.

No matter what "Spring Forward" actually refers to, Apple is slated to start delivering the Apple Watch in April. By having a prerelease media event to show it off in more detail in March, as well as introducing a pre-order date before shipping starts, Apple will gain another big news cycle.

While the new Pebble Time smartwatch broke pledge goals on Kickstarter, rocketing up to more than 51,000 backers and US$11.2 million in funding with 28 days to go, the Apple Watch likely will break waves, compared to the ripples made by Pebble Time. So yeah, Apple will be well served to give potential buyers a little heads up before they flood Apple Stores. More to the point, Apple needs to give new buyers a chance to figure out which version of the Apple Watch they want to wear -- not to mention figure out which Apple Watch they can afford.
So What's the Big Mystery?

As near as I can tell, the Apple Watch event likely will fill in some knowledge gaps around functionality, as well as pricing -- and while I'll get to them in a minute, the mystery is whether most of us new buyers suddenly will turn into watch wearers for longer than a few months.

Will I become a watch wearer again?

I don't know. Oh, I'll snap up an Apple Watch as soon as I can, probably a Sport Edition, but that doesn't mean that I'm sure I'll like it and use it everyday. There are two reasons I'll buy, though.
First, I write about Apple, so I'm buying an Apple Watch. Second, even if I didn't write about Apple, Apple is offering an interesting leap forward in smartwatch technology, especially for Apple enthusiasts. Because I would like to have an iconic everyday watch but haven't yet been enticed enough to buy any, an Apple Watch is a likely choice.

The mystery is whether an Apple Watch will be useful enough to be worth wearing... and charging... every day, especially for people like me -- Apple enthusiasts who buy Apple gear because we appreciate using it even more than we like its looks.

For us, the Apple Watch will need not only to fit our personalities, but also to offer us a reason for being on our wrist in the first place. I've already written a bit about how Apple will manipulate our buying triggers and get us to purchase an Apple Watch... but loving it for months and years? That's still a mystery.
Hopes, Dreams and Limitations

What can we expect to learn from Apple's March 9 event? Obviously, pricing. The Apple Watch (basic edition) starts at $349 but the Apple Watch Sport and Apple Watch Edition options could retail at any price. Seriously, pick any number over $449 and less than $21,000 you might not be wrong.

Other factors also may become more clear, however, including initial limitations. For instance, because the Apple Watch must pair with an iPhone for location tracking -- in order to use the iPhone's GPS and WiFi abilities -- the Apple Watch isn't going to map your run and tell you exactly how much distance you traveled while trail running -- unless you also pack your iPhone for your run.

For some, this is a big disappointment, but apparently it's not that big a deal for the two very fit young ladies featured in Apple's health and fitness promotional video. (If you can spot their iPhones hidden in their body-hugging workout clothing while they run around outside, let me know.)
As for me personally, I would be interested in knowing how many miles I cover when I play basketball for an hour. Also, while I could tape over the Apple Watch or slip a wristband over it while playing ball, will it maintain a connection with my iPhone when it's off the court? Or would I have to strap the iPhone to my body, too?

To be fair, you can't expect a gadget the size of an Apple Watch to do everything -- there's a give and take, right, and we understand that. It just means that the Sport might not be quite as sporty as a super active person would hope.
Charging, Charging and More Charging

One huge question concerns charging: Will a normal person be able to get through a day wearing an Apple Watch? What if that day is Friday with a Friday night out on the town?

How long will it take to charge? Will you need to charge it every night? It's highly likely, which means you might not be able to use your Apple Watch to help monitor the quality of your sleep. If there's one thing that many people could improve on, it's sleep habits.

I'm guessing I won't be able to use it as a quiet vibrating alarm for when I need to get up really early -- unless I give it a precharge before bed.

If charging becomes a high-maintenance chore, will the Apple Watch still help me live a better life? I don't know -- I have the nagging feeling that great technology should free us, not tie us down.
Is It Waterproof?

Recent comments by Apple CEO Tim Cook suggest an Apple Watch can be worn in the shower, which is great and all. If I have to charge it every night, though, I'll hardly need to put it on right after I get out of bed and before I step into the shower.

Right now, the Apple Watch Sport seems like it's only for sissy indoor, simulated sorts of sports. Does shower-proof mean it can handle three hours of drizzling rain? Sport my ass.

I'm interested in what it can do in deeper water. Can you go hot tubbing with it? How about swimming? Snorkeling is probably out. What about paddle boarding or kayaking? What's your all-day risk look like? Can it be dunked completely down to six feet or so multiple times? Or does one underwater crash mean your Apple Watch is done?

These are some questions that I hope Apple answers before the Apple Watch goes on sale.
What About Apps?

Some reports have indicated that the first round of apps will allow only basic features -- for instance, no pressure touch and no taptic vibrations.

Despite limitations that likely will be put in place to ensure consistent experiences -- and make sure apps don't drain the battery too fast -- I believe Apple will share the stage with a trusted third-party app developer or two to show off some interesting new app.
Apple Watch Accessories

Meanwhile, other reports indicate that you shouldn't get your hopes up for the ability to buy third-party custom watchfaces anytime soon. If you want Mickey Mouse, great, but something entirely different from a visionary third-party designer? Maybe not soon.

How about those bands? It's still unclear as to how Apple will price its own bands, but what I'm really curious about is how Apple will allow third-party bands. Will they approve them? Certify them? Or will manufacturers just start building their own?

I don't care how it happens, but I sure as heck want to see a wider, more manly band. Just saying. Something overly manly, even. If I can't find a non-Apple band that fits, I'll have to fashion my own out of barbed wire or modify a Leatherman Tread band to make sure that the world, in addition to myself, knows that I'm still packing plenty of testosterone.
Something New and Special?

I'm holding out some hope that Apple will introduce a new Apple TV home automation hub device and that the Apple Watch will play a role, but I think the Apple Worldwide Developers Conference will be a more likely venue. So what, then, anything?

I think so.

I'm having a hard time imagining an hour-long rehash of the Apple Watch introduction without introducing something new -- or other Apple products, too. The Apple Watch is certainly important enough to Cook and Apple to warrant an hour-long pitchfest, if not longer. Yet I can't help thinking Apple has more of a story to tell than we think we know.

Of course, if you can get out of regular work hours March 9, you can view a live stream of the event via OS X with Safari, or iOS or an Apple TV.