Some call it: In-Vehicle Experience of the Future. I call it Helga. Say ‘Hello’ to Helga, my car’s AI.
‘Emirai’ system Mitsubishi
designboom.com
Helga: “Good morning, girl! Are we going to work? I’m 96% charged so I’m good to go. I sense you got your coffee mug, it’s about 70% full, so not stopping at Starbucks.”
Me, settling in: “Morning! Yes – work, no Starbucks, and I have to drop off a package in the mail.”
Helga, pulling out of the driveway: “Nearest post office is still closed. We can stop by at the one near your office, by the time we get there it will be open. Traffic looks bad this morning, so I’m not taking the freeway, re-routing via expressway. ETA 8:43, that’s in 36 minutes including 9 min at the post office, I don’t know what’s the hold up, but there’s a line. You’ll still have enough time before your 9:00 meeting with your boss and 5 peers titled ‘Helga Nav Update UI review’. Hey, wait-a-minute… Helga – that’s me. Am I getting an upgrade?”
Me, smiling: “Yes, you lucky beast, you’re getting an upgrade. Now enough with the chit chat. Route approved. But I do feel like driving today. Do you mind?”
Helga: “Well, so long as you don’t go over 90 MPH. I do not enjoy the anxiety. And watch out for that pole, when we arrive at the parking garage. You almost hit it last time. No need for name-calling. If I’m a beast, at least I’m a pretty one. You got the wheels?”
Me: “I got the wheels. Hey, don’t mention my past behavior when there’s a passenger with us. Play us some Vivaldi?”
Helga: “Acknowledged. What Vivaldi, I thought it’s a Maroon 5 kind-of-day”.
Me: “No, I want some classic. My 9:00 is going to be very stressful; I want to clear my head”.
Helga: “Your wish is my command, Vivaldi it is. You are so unpredictable.”
Me: “Oh, I’m so lucky that is not true. If it were true you would have been a pointless impossibility.”
———————————————————————————————————————
So what do you think of the above conversation, between me and Helga? Whatever you think, you should know that it is absolutely, positively, possible and feasible with technologies that exist today.
1. After decades invested in research, multiple companies and research institutions can provide speech recognition by recording voice, analyzing the audio data, and transcribing it to text, using language models. The more focused the usage and the language model (on a domain or specific accent), the higher the accuracy.
2. Next step is understanding the text. A wealth of available NLP (Natural Language Processing) technologies can decipher human language, analyze it, and respond in a meaningful way according to rules learned from vast corpus of training data. Apple’s Siri and Google’s Now are both based on these technologies.
3. A computer generated voice can be routed to the car’s speakers. Cars had the ability to ‘talk’ some limited sentences such as ‘Lights are on!’, in some models since the 1990’s, but I suspect that might have been a recording. That’s not the same as generating the audio output on the fly.
4. % charged – well, the car ‘knows’ exactly what’s the status of the battery, or if it is gasoline based it can tell tank % full.
5. The presence of the coffee mug – a simple sensor for weight or even heat can sense it and make use of this info.
6. All route and re-routing calculations mentioned – Waze and other map apps already do all of this, just need to verbalize it.
7. Self driving? I don’t think I have to tell you about all the self parking cars that are already available in the market today, and all the self driving prototypes in testing. Cameras and sensors all around the car make it possible to identify proximity to obstacles and other moving vehicles, to give it a complete picture of the environment in which it operates. Machine learning is used to learn how drivers operate in these conditions. It’s a very complex problem to solve and it will take years, but it will be solved.
8. Car apps: If your phone knows your schedule – there’s no reason why your car can’t have access to it.
This is it. This is where devices like ‘CarPlay’ come into play.
Last month was special: 3.14 – Pi month! And it was also special because Apple CarPlay was announced, a device that connects your iPhone to your car via an iOS display, that brings you control over messages, music, and navigation. Eventually more and more apps will be adapted to this use-case and be brought into this offering. The idea is that people have dozens of applications running on their phone, and this is a way of adapting them and bringing them into the car.
Apple CarPlay in Volvo
Autoblog.com
The changes in this area are happening at the speed of light. In 2008-9 in many cars you could listen to music played off your iPod or iPhone using an auxiliary cable, or connect your blue-tooth enabled phone to the car, to have conversations and music through the car speakers. This was much better than the clunky car-phones that existed only in luxury cars just a few years earlier. In 2012 many cars already have a USB connection, that allows you to also charge the phone, while browsing through its music using the car’s controls. But the car – phone integration was not yet complete, and was not the only thing happening in the cars.
The truth is this space is messy. While there is an attempt to have better interfaces between cars and smartphones, to take advantage of apps you already have, there is also an attempt to create a platform of both hardware and software to run dedicated apps in the car. If you try to somehow mix the two, you also have a problem of compatibility. Think about it this way: every manufacturer is creating their own platform, and then some soliciting developers to develop apps for them, some developing their own. 2013 saw a burst in car-based applications, including a competition for developers conducted by Ford. Some manufacturers use satellite based wireless internet. Many rely on Internet connectivity via cellular networks of users’ smartphones, but that means users have to either switch car or switch phone if they are incompatible. This became clear with the Porsche launch at the end of 2012, with platform integration for Blackberry.
This is still an issue with CarPlay. Until a similar Android-based device is introduced, you would get the system with your new 2015 Mercedes-Benz, Volvo or Hyundai only if you own an iPhone. It’s interesting to see it implemented in different ways in the 3 different cars. Here’s a video showing its introduction at the Geneva Auto show 2014.
Will a standard platform be created, as an option for providing applications in any car, and connecting the apps from any phone? Will car apps ignore phone apps or become superior for in-car use? It seems 2014 is the year of platforms and connectivity alliances. Car manufacturers enter relationships with wireless internet providers, and a new alliance (OAA) was created to bring the Android platform to connected cars. Only time will tell us what to make of this.
What I do know is that as far as the future of car apps is concerned, CarPlay is just the tip of the Iceberg. A real giant Iceberg.
Kidmat Eden 
Some comments provided by email:
This is similar to work done at Toyota ITC.
First thing I notice is the chatty dialogue. This needs to be done very carefully. For demos it needs to be bold, but for real usage we found that it needs to be subtle. And the tone of voice needs to match the driver’s current state. A sad driver hates a happy voice so much that they actually have more accidents
http://dl.acm.org/citation.cfm?id=1057070
It is useful for keeping the driver awake and aware … but only if it’s interactive, requiring engagement from the driver
http://hfs.sagepub.com/content/50/5/772.short
And, word choice is important especially if you’re informing the driver of a mistake
http://dl.acm.org/citation.cfm?doid=985921.986028
> “Play us some Vivaldi?”
There are many studies showing that the type of music can influence your driving, and the quality of office work, and mood. I’ve never seen it applied to driving, though. But the long term trend for drivers with long commutes is different: they care less about entertainment and more about “getting work done” during the “perceived downtime” in the vehicle. They’d much prefer to read email, read or hear news (personalized), ping their employees, set up meetings than to hear some Vivaldi. This is a long term trend over at least 15 years, reported by Nationwide Insurance and Hartford and many others.
> the car ‘knows’ exactly what’s the status of the battery …
It knows a lot. In fact “it knows” was a central theme buzzword at Toyota ITC. Working on in-vehicle displays “for the purpose of reducing Range Anxiety.” – one of the ideas was that, if “the car” could know about the availability of charging stations nearby, it could adjust its fuel usage (electric vs. gasoline) optimally.
“Optimization” is another buzzword. The idea is that, in the future, if they have a car with 3 types of fuel (gasoline, battery, Hydrogen-Fuel-Cell) then they could modify the usage of each type based on various factors. I worked on optimizing it by altitude of the route. If we know the driver will soon be driving downhill, then we should deplete the battery if possible. Reason: the battery “overfills” after just a few minutes driving downhill.
We could also do it by time of day (if solar power), temperature (battery is inefficient in cold weather), air pressure (already doing this for gasoline/air mixture). Basically optimize the engine on as many factors as possible.
> Usability:
The vimeo video shown at the end of this page is perfect for the Auto Show, and it even has a bright screen so it could work when the sun is shining. But it would be panned at the Automotive UI conference. The phone icon is such a tiny target. If the car were moving, could you hit it reliably? What if you mistakenly hit something else, would you need to divert your attention to undoing your mistake? Is it a capacitive screen? Does it work if you have gloves on? Is the glance time under 1.5 seconds for each function?
I bet that the popularity would increase if you could get some real work done: it reads your emails in a good-sounding voice, or your calendar appointments. Also your idea of integrating business hours, waiting times, and traffic conditions is really great. How about this:
“Because there is a traffic jam on 101 where 85 joins, I recommend you exit at Ellis Street, eat a breakfast at Specialtys Sandwiches, and 30 minutes later the traffic will probably be clear.”
By the way I really liked the “watch out for that pole, when we arrive at the parking garage. You almost hit it last time.” If it could warn based on past behavior…
Here’s another that I haven’t heard that anyone patented. Driver says “Before going home I need to buy a shower curtain.” Current time is 9:55pm. Who is open, that sells shower curtains? Maybe only one little convenience store at a 24 hour gas station. I would want to know about it. What if that convenience store owner used an Oracle database for inventory control, and the Navigation system could connect to it – wouldn’t that be convenient? If they knew I was desperate to buy a shower curtain TONIGHT, maybe they could push up the price $1. Or $2. Or $10 or $20?
What if, instead of a shower curtain, the item I want is a battery charge? Maybe the charging station would want to negotiate with me? But I’m driving, I don’t want to be bothered with negotiating. What if the system could negotiate fairly between vendor and customer? Have you ever heard of such a system? I actually have, and it was related to EV charging.
LikeLike