AWS machine learning in art sparks beautiful logic

Perhaps the first lightbulb to ever be switched on publicly in the United States capital was at the Smithsonian Institution’s Arts and Industries Building for President James Garfield's inauguration. The demonstration of Edison's invention wowed the audience and changed the absolute fabric of civilization from that day forward. A century and a half later, architect and designer Suchi Reddy has built, in that same space, a two-story light-filled sculpture titled me + you. Commissioned by Amazon Web Services (AWS), visitors speak a single phrase, “My future is…” and then fill in the blank with a single word. The entire phrase is “heard” by the piece. Using Amazon's machine learning (ML) and language recognition systems, the sculpture's code responds with a physical display of kinetic color and light that expresses those uttered visions of the future. me + you points us toward entirely novel ways to put machine learning to use, and it is one example of the possibilities that are presented when technology and art collide.

Swami Sivasubramanian, who heads up machine learning at AWS, sat down with Suchi Reddy to discuss her work and the future of art and machine learning. In turn, Reddy had her own questions for Sivasubramanian about where he sees ML headed next.

So, tell us. How did you initially approach this sculpture?

Well, I'm an optimistic person, and I wanted to study how we're really evolving with technology. I want to think about technology in a place where it really makes us better humans. It had to be about people being able to voice their feelings about the future. In all of my work, whether it's architecture or design, I believe that form should follow feeling. When we can express how we feel in the world, we can influence the world to help us become better people. I was thinking about that.

It's an intersection of technology meets art. I'm fascinated to know how you mixed art and design and industrial engineering. What did it take to build this?

It's been an amazing process over the last two years, working with the professional services team at AWS. They've been so good to work with. I started with no real knowledge of how artificial intelligence can work. I started with a drawing and visualized it in the space. I worked with the height and the size, and how it needed to feel, and what the quality of the interaction needed to be. Then, I needed to figure out how to translate that two-dimensional object to three dimensions and give it this emotional heart. It became this dialog with technology.

To work with your team, and to work with my coding team, we really looked at how we can take voice and turn it into light, and turn that light into mandalas that are particularly unique and specific to the person who is speaking into them. The sculpture understands our pitch, our sound, and our emotions. As artists, we are the mediator of that conversation. This is what I love about what I do. It's never one thing. It never goes linearly.

When I first talked about the work, the engineering team was a little perplexed. We weren't running sprints. I learned from them, and they learned from us. They got to be fluid with it. Also, the physical engineering of it—filling this space with light. Taking the energy of sound and turning it into the energy of light was really what I wanted to do. And, I wanted to make it organic. I wanted it to feel like a living thing, not an arrangement of dots.

I never heard an artist say that they had a coding team until today. But this art is very intriguing in the amount of engineering and design that went into it. From the optical cables, the mandalas, and converting it into an engine that listens and understands, and converts it into light. It's a great blend.

You'll notice it takes the form of a lingam [a votive that symbolizes the Hindu god Shiva] and a kind of presence rising in a space. Which I thought was also important, because we speak our individual visions of the future into, what I'm calling here, the cloud, or the base. But the central totemic part puts it all together. It creates a record––through my eyes––of all of our visions of the future. How we are all evolving together. It's our dream to slice that by day and by year, and keep it going to see how that changes.

Where I grew up in South India, textiles are a big part of our culture. The looms were some of the oldest computers ever designed. That leads to this central totemic part. I was thinking of it as weaving a light textile. Really making it feel like a loom. So much of South Indian history is told through our textiles, and you can see the evolution of craft. You can see the evolution of color and political influences. That happens in Africa, that happens in India––it's a record of our lives, and I really tried to bring some of that language together into the central piece.

The space we're in exhibited one of the first electric lightbulbs. How do you see this piece bridging that gap?

I'm really happy no one told me this is the spot where Edison unveiled his first lightbulb when I started, because I probably wouldn't have gone in the direction of a light sculpture. But I came in, looked at the space, and I felt it had to be about light. This rotunda brings in so much light, and physically, the idea that humans are always moving towards the light, is something that's very deep for me and my culture. The rotunda is this circular connector, and the sculpture really took shape because of the space.

I'm excited about innovating with new things. Like you were saying, you'd never heard of an artist with a coding team before. I didn't either. But for me, code is a new material. I really learned through this process that every day we have new technologies that are new materials to work on.

What has machine learning come to mean to you? Comparing what you knew before to now, having used machine learning and artificial intelligence to create such an amazing piece.

Over the last two years and particularly in working with your team, I've grown to have a huge respect for machine learning as a system or a structure through which we understand our world. I think if we can use data as a lens, if data becomes another material, then it depends how you focus that lens. If we focus it toward equity, empathy, diversity, or expression, we can focus it in a way that makes full use of the potential. That was the deepest learning and lesson for me.

In my lifetime, I feel that change has accelerated more than it did in my parent's lifetime. I'm sure in my nephew’s and niece's lifetime, it will be even faster. To understand how all that fits into our values, and how we align those values with what we're making (and how we make it), instead of being on some kind of runaway train that's just going to take us where it wants to go––that was all really part of the piece too. To highlight human agency in technology.

When you ask someone to give you a word about the future, they stop to think. That really is what the is sculpture is about. You want people to have a self-awareness about what the future is going to be.

But machine learning is a huge field. I want to know how you see this as a force in our society. How do you see machine learning being this vehicle that carries us forward?

Machine learning is one of those technologies that has such transformative potential—accelerating drug discovery or producing vaccines, helping customers get better health care, improving how goods move through a supply chain, accelerating loan applications, automating aspects of everyday life. And it's just getting started. Humans have always been the creative kind. And we want to automate the tasks we find routine and boring. This is where machine learning and artificial intelligence come in. Humans used to manually switch and connect lines for telephones when one person called another. In the same way, machine learning will remove the boring, repetitive tasks.

My personal mission is to make this technology accessible to all builders in the world. You don't need a Ph.D. to use this technology. That's what excites me.

That's what excites me, coming from India, where I saw my country switch and leap because of technology. If you are putting this technology in the hands of anyone, you are giving them so much more than they started with. That's your goal with accessibility?

India is a good example here. In the late 1990s, technology changed the entire workforce and made it tech-first. But all the changes and promises with machine learning and artificial intelligence are not possible unless we get these technologies into the hands of developers or front-line workers who work in the areas that need improvement. We, as an industry, must make these magical technologies—then make them accessible to everyone and anyone putting the technologies to use.

In my experience working with it, I truly felt I was watching magic. You're creating something from a desire to know more, which is basically human. That's what's so inspiring to me, working with this idea of the collective. This is the right tool for that. It looks at how we function collectively. The bigger the dataset, the more information you have. The more you train the model, the more you can understand your world. If you really look at machine learning as a powerful tool to learn about us, then what do you think it can teach us?

At the end of the day, I view it as a tool that builders and creators are going to use. To me, the more interesting question is what it will let us build to make us better. It's all around us, not only in recommending movies and how to shop, but you don't have to wait as long at a call center. Routine questions are answered faster. It's transforming the health care industry and pharmaceuticals, and how fast they move information. I hope when we look back 100 years from now machine learning will be as pervasive as electricity.

This was a creative experience, working with code. What do you see as the creative aspects of machine learning? Do you think code can be creative?

Many software engineers say code is art. It is not about what you write, it's about how. They are very passionate about how code gets built and how it is maintained. You might deeply care about how they maintain your art for years to come. Many engineers I work with have a long-range view of how their code will be used. They view themselves as artists.

I personally view code as one of the ultimate manifestations of how logic meets creativity. Creativity cannot always be logic. Logic doesn't always come first. You can't tell an artist: Create this. But with code, you have a medium to transform ideas. That's what the process is about.

I had such a respect for the engineers on the team. They had such a passion. You can think about it as advancing aesthetics, or experience. The more I look at it, I see a world inside coding that contains all of the seeds for creating something magical and interesting. But what, in your mind, should someone take from that? Should machine learning be taught in schools at a young age, or should it only be in the hands of professionals?

Machine learning is still in its infancy, in human terms, toddler-age. But it is a powerful tool that will be an important tool for everyone to know and learn from their early days of school. Maybe not elementary school, but when they learn math and science. As we are expected to use computers in our daily job, I expect we'll see the same with machine learning systems, even if you don't write code. It will be an interesting toolkit for kids to know.

When I came into my project I felt AI and ML were around us, but we hadn't yet become conscious of it. Maybe that happens with every technology. The inventors show the world what's possible, and then it takes on a life of its own. So maybe when machine learning is a teenager, we will have to start implementing rules. How do you see machine learning growing?

By putting it in the hands of all builders, one of the most important things we need to do is make sure they use it in a responsible fashion. This is an area that my team, the industry, and academia are considering. What kinds of guidelines do we need to institute? We have guidelines for electricity. You don't think about them when installing a lightbulb, but you do when you build high-voltage wires. These are some of those things that society will be thinking through as this technology goes mainstream. The important thing is that you want technology as transformational as machine learning to be able to solve hard problems. You don't want to limit it. So you have to think about these problems in a careful manner.

Now, I have to ask. Your firm does design and architecture, not just sculpture. How has working with machine learning changed the way you'll think about your future builds?

I've been thinking about it a lot throughout this process. I'm lucky to work in lots of different ways, and bring them together. I want to be on the leading edge. But I think about how machine learning can help us transform the environment. There are such big challenges right now. Climate change. Sustainability. We can be so much less wasteful in building. There is a huge potential for machine learning to inform us about how to best manage our construction processes. To apply the beautiful logic I've seen to create this sculpture to how we build our buildings could be amazing. It could actually free up a lot of energy.

But also, as I've said, I want to really use these tools to look at equity and empathy. If you can look at how to make things more equitably accessible to people, in the world, whether public space, public art, or access to buildings. If we can use machine learning to see how people can really appreciate that access and opportunity, and at what level its useful to them, that could be truly amazing. It is fascinating to me. I can't wait actually.

me+you is on display in the Smithsonian Arts + Industries Building as part of the Futures exhibit through July 6, 2022.