AI in the things we make
“Help me write” in Gmail
There are a lot of great ways that creative AI is helping to improve our products, and Gmail is a great place to start. In 2017, we started offering Smart Reply, a set of short answers that you could choose with just one click. Then came Smart Compose, which made ideas for what you should write as you typed. With the help of AI, Smart Compose led to more powerful writing tools. They have been used more than 180 billion times in Workspace in just the last year. And now, with a much more powerful model, we’re taking the next step in Gmail with “Help me write.”
Say you got an email telling you that your flight had been cancelled. Even though the company sent you a voucher, you really want a full refund. You could answer with “Help me write.”
Just type in what you want, like “an email asking for a full refund,” hit “create,” and a full draught will show up. It conveniently pulls in flight information from the previous email. It looks pretty close to what you want to send, but maybe you want to improve it further. In this case, a more detailed email might make it more likely that the return will come through. As part of the changes we’re making to Workspace, “Help me write” will start to be used. And just like Smart Compose, it will get better as time goes on.
New Immersive View for paths in Maps
Since the early days of Street View, AI has stitched together billions of panoramic pictures, so people can explore the world from their device. Last year at I/O, we showed off Immersive View, which uses AI to make a high-fidelity version of a place so you can experience it before you go there.
Now, we’re using the same technology to help you get where you want to go, which is what Maps does best. Every day, Google Maps gives directions for 20 billion km, which is a lot of trips. Now imagine if you could see your whole trip in advance. Whether you’re walking, driving, or riding a bike, you can use Immersive View for routes.
Say you want to go for a bike ride in New York City. Maps has given you a couple of options close to where you are. The one along the water looks nice, but you want to get a feel for it first, so you click on Immersive View for routes. It’s a whole new way to think about your trip. You can zoom in to get a great view of the ride from above.
There is also more information out there. You can look at the weather, traffic, and air pollution to see how they might change.
Immersive View for routes will begin to roll out over the summer, and launch in 15 places by the end of the year, including London, New York, Tokyo and San Francisco.
A new way to use Magic Editor in Photos
Google Photos is another thing that AI has made better. We showed it off at I/O in 2015, and it was one of our first products built from the ground up to use AI. Machine learning has made it possible to look through your pictures for people, sunsets, or waterfalls.
We want you to do more than just look for photos, though. We also want to help you improve them. In fact, 1.7 billion photos are changed every month in Google Photos. AI developments give us more powerful ways to do this. For example, Magic Eraser, which was first released on Pixel, uses AI-powered computational photography to get rid of things that don’t belong in a picture. And later this year, you’ll be able to do a lot more with a new experience called Magic Editor. It uses both semantic understanding and creative AI to let you do a lot more.
Here’s what I mean: This is a great picture, but as a parent, you probably want your kid at the center of it all. And in this one, it looks like the balloons got cut off, so you can move the party boy. Parts of the bench and balloons that were not in the original shot are instantly made by Magic Editor. You can punch up the sky as a final touch. This also changes the lighting in the rest of the picture so that the edit looks like it was done all at once. It’s truly magical. We can’t wait for Magic Editor to come to Google Photos later this year.
Making AI more helpful for everyone
AI can help you with things like Gmail, Photos, and Maps. These are just a few examples. And there’s so much more we can do to make sure the goods you know and love use AI to its fullest.
Today, we have 15 products that each serve more than half a billion people and companies. Six of them have more than two billion users each. This gives us a lot of chances to fulfil our goal, which is to organise the world’s information and make it useful and easy to find for everyone.
It’s a task that will last forever and seems more important every year. Looking ahead, the most important thing we can do to move towards our goal is to make AI useful for everyone. We’re doing this in four important ways:
First, by learning more and getting smarter, and by getting a better grasp of the world.
Second, by making people more creative and productive, so they can be themselves and get things done.
Third, by enabling developers and businesses to build their own transformative products and services.
Lastly, by building and using AI in a responsible way so that everyone can benefit.
Gemini and PaLM 2
We’re so excited about the chances that lie ahead. We’ll be able to make AI useful for everyone if we keep improving our base models. So I’d like to tell you how we’re going to deal with them.
We talked about PaLM last year, which led to many improvements across all of our goods. Today, we’re ready to tell you about our newest PaLM model, PaLM 2.
PaLM 2 is based on our basic research and our most up-to-date systems. It can do a wide range of tasks well and is easy to set up. Today, we are introducing more than 25 PaLM 2 products and features.
PaLM 2 models come in a wide range of sizes and all have good foundational powers. We’ve affectionately named them Gecko, Otter, Bison, and Unicorn. Gecko is so light that it can run on mobile devices. It’s fast enough to run great interactive apps on the device, even when it’s not connected to the internet. PaLM 2 models are stronger in logic and reasoning thanks to broad training on scientific and mathematical issues. It has also been trained on text in more than 100 different languages, so it can understand and produce nuanced findings.
PaLM 2 has strong coding tools and can also help developers work together from all over the world. Let’s say you’re working with a colleague in Seoul and you’re debugging code. You can ask it to fix a bug and help out your teammate by adding notes in Korean to the code. It first recognizes the code is recursive, then offers a fix. It tells why the fix was made and adds the Korean comments you asked for.
PaLM 2 is very powerful, but it really shines when it is fine-tuned based on knowledge of a specific topic. We just put out Sec-PaLM, which is designed for security use cases. It uses AI to better identify malicious scripts, and it can help security experts understand and resolve threats.
Another example is Med-PaLM 2. In this case, it’s adjusted based on what doctors know. When compared to the base model, this fine-tuning made the reasoning 9 times less likely to be wrong, which is close to the performance of clinician experts who answered the same set of questions. In fact, Med-PaLM 2 was the first language model to score at “expert” level on medical licensing exam-style questions, and is currently the state of the art.
We’re also working to add features to Med-PaLM 2, so that it can synthesize information from medical imaging like plain films and mammograms. You could imagine AI working with radiologists to help them understand images and share the results. Here are some ways that PaLM 2 is used in specific fields. We can’t wait to see it used in more, which is why I’m pleased to announce that PaLM 2 is now available in preview.
PaLM 2 is the latest step on our 10-year journey to bring AI to billions of people in a responsible way. It builds on progress made by two world-class research teams, the Brain Team and DeepMind.
When you look back at the most important AI advances of the last 10 years, you can see that these teams were involved in a lot of them, like AlphaGo, Transformers, sequence-to-sequence models, and so on. All of this helped us get to the turning point we’re at now.
We recently brought these two teams together into a single unit, Google DeepMind. Using Google’s computing power, they are focusing on making safer and more responsible systems that can do more.
This includes our foundation model for the next generation, Gemini, which is still being trained. Gemini was built from the ground up to be bidirectional, very good at integrating tools and APIs, and ready for future innovations like memory and planning. Even though it’s still early, we’re already seeing amazing multimodal features that weren’t available in earlier models.
Gemini will have different sizes and powers, just like PaLM 2, once it has been fine-tuned and thoroughly tested for safety.
AI responsibility: Tools to identify generated content
As we invest in more capable models, we are also deeply investing in AI duty. That means having the tools to spot content that was made by a computer when you see it.
Watermarking and metadata are two important ways to do this. With watermarking, information is put right into the content in a way that stays even after simple image editing. Moving forward, we’re building our models to include watermarking and other techniques from the start. If you look at a synthetic image, it’s impressive how real it looks, so you can think how important this is going to be in the future.
Metadata lets people who make content add more information to the original files. This gives you more information whenever you see an image. We’ll make sure that every picture made by AI has that metadata.
Changes to Workspace and Bard
As models get better and more capable, one of the most exciting opportunities is making them available for people to connect with directly.
This is the chance we have with Bard, our conversational AI experiment that we started in March. We’ve been fast evolving Bard. It can now be programmed in a lot of different ways, and it is much better at thinking and maths questions. It is now fully running on PaLM 2 as of today.
Google Workspace is also getting new tools. In addition to “Help me write” in Docs and Gmail, Duet AI in Google Workspace provides tools to generate images from text descriptions in Slides and Meet, create custom plans in Sheets, and more.
Introducing Labs and our new Search Generative Experience
As AI keeps getting better quickly, we’re focusing on giving our users features that will help them. We’re also giving you a new way to try out some of the features in Workspace and other products starting today. It’s called Labs. I say it’s new, but Google has been using Labs for a long time to let people try out new things early and get feedback. You can start signing up later today.
Alongside the Workspace features you just saw, one of the first experiences you’ll be able to test in Labs involves our original product, Google Search. The reason we began deeply investing in AI many years ago is because we saw the opportunity to make Search better. With each new idea, we’ve made it more useful and easy to use.
Improvements in language understanding let us ask questions more easily and reach the most relevant content on the web. Advances in computer vision brought new ways to search visually. Now, even if you don’t have the words to describe what you’re looking for, you can search anything you see with Google Lens. In fact, Lens is used for more than 12 billion visual searches every single month. This is a 4X rise in just two years. Lens and multimodality came together to make multisearch, which lets you search with both an image and text.
As we look ahead, Google’s deep understanding of information combined with the unique capabilities of generative AI can transform how Search works yet again, unlocking entirely new questions that Search can answer, and creating increasingly helpful experiences that connect you to the richness of the web.
Putting generative AI to use in Search is, of course, still in its early stages. People around the world rely on Search in important moments, and we know how critical it is to get this right and continue to earn their trust. Our North Star is always that.
So we’re approaching innovation responsibly, aiming for the highest bar for information quality as we always have from the very beginning. Because of this, our new Search Generative Experience will be available first in Labs.
Making it easy for others to create new things
AI is not only a powerful enabler, but also a major change in the way platforms work. Every business and group is thinking about how to drive transformation. We want to make it easy and scalable for others to use AI to come up with new ideas.
That means providing the most advanced computing infrastructure — including state-of-the-art TPUs and GPUs — and expanding access to Google’s latest foundation models that have been rigorously tested in our own products. We are also working on providing world-class tools so that users can train, fine-tune, and run their own models with enterprise-grade safety, security, and privacy.