“As Artists we should be constantly evolving our technical skills and thought processes to push the boundaries on what's achievable,” Marco Matic Ryan, Augmented Reality Artist

Augmented and Virtual Reality is taking on the world, one AR app at a time. Almost all tech giants have their own AR platforms, whether it be Google’s ARCore, to Apple’s ARKit, to Snapchat’s Lens Studio. Not just that, there are now various AR galleries and exhibits focused solely on AR, where designers and artists showcase their artistic talents combined with rich augmented reality experiences.

While searching for designers and artists working the augmented reality space, we stumbled upon Marc-O-Matic aka Marco Ryan. We were fascinated by his artwork and wanted to interview him right away.

[embed]https://vimeo.com/219045989[/embed]

As his website bio says, “Marco is a multidisciplinary Artist, Animator, Director, Storyteller and Technologist working across Augmented and Virtual Reality technologies.” He shared with us his experiences working with Augmented Reality and told us about his creation process, his current projects, tips and tricks for a budding artist venturing into the AR space and his views on merging technology and art.

Key Takeaways

A well-rounded creative is someone who could do everything from creating the art, the associated stories as well as execute the technical aspects of the work to make it come to life. The future belongs to these creative types who love to learn and experiment every day.

An artist must consider three aspects before taking on a project. First, how to tell their story in a more engaging and interesting manner. Second, how to combine different skill sets together and third the message their work conveys.

Augmented Reality has added a new level of depth to the art experience not just for artists but also for viewers. Everyone has a smartphone these days and with AR you can add many added elements to an art piece. You can add sound, motion and 3D elements to the experience, which affect more of your senses.

It is easy for a beginner artist to get started with creating Augmented reality art. You need to start with learning a basic language (mostly C# or Javascript) and then you can learn more as you explore. Tools and platforms such as Adobe After Effects, Maya and Blend, are good for building shaders, materials or effects.

Unity 3D is a popular choice for implementing the AR functionality, and over 91% of HoloLens experiences are made with Unity. It supports a large number of devices and also has a large community base. Unity offers highly optimized rendering pipeline and rapid iteration capabilities to create realist AR VR experiences.

AI and Machine Learning are at the forefront of innovating AR/VR. Machine learning algorithms can be used in character modeling. They can map your facial movements directly to a 3D character’s face. AI can make a character emotionally react based on the tone of the audio and also automate tasks allowing an artist to focus solely on the creative aspects.

Full Interview

How did you get started in AR. How did you journey from being an artist to an AR artist began?

I started experimenting with Immersive AR/VR tech about 2-3 years ago? At the time buzzwords like virtual and augmented reality started trending and it had me curious to see how I could implement my own existing skills into this area at the time.

Prior to getting into AR and VR I came from a largely self-taught background in Art & Illustration which then evolved into exploring film and animation and then finally led to game design and programming. Exploring all these areas allowed me to become a well-rounded creative, meaning I could do everything from creating the art, the associated stories as well as execute the technical aspects of the work to make it come to life.

That’s quite impressive and fascinating to know! What programming language did you learn first and why? What was your approach to learning the language? Do you also dabble in other languages? How can artists overcome the fear of learning technology?

Working with the game engine Unity 3D to create my VR and AR Experiences, the engine allows you to program in either C# or Javascript. C# was the the most prevalent language of the two and for me I found it easier to pick up. As a visual learner, I initially came to understand the language through node based programming tools like Playmaker and Uscript. These plug-ins for Unity are great for beginners as they allow you to visually program behaviors and functionality into your experience by creating interconnected node trees or maps which then generates the code.
I’m familiar with this form of logic building as other programs I use, such as Adobe After Effects, Maya and Blend, use similar systems for building shaders, materials or effects.

Through node based programming or visual scripting, you can take a look under the hood and understand the syntax iteratively. It’s like understanding a language through reverse-engineering. By seeing the code being generated based on the node maps you create you could quickly understand how the language is structured.

I try to think of learning new things as adding skills to your arsenal and the more skills you possess the greater you can expand on your ideas. I don’t think it’s so much a fear of learning new technology. I think it’s more a question of ‘Will learning this new technology be worth my time and benefit my current practice?’ The beautiful age we live in makes it so much easier to learn new skills online. So there’s so much support already available for anyone wanting to explore game design technologies in their creative practice.

What tools do you use in the process and why those tools? What is the actual process you follow? What are the various steps?

The Augmented Reality Art I created has two parts to them:

Creating the Art

Everything is drawn on paper first using your typical office ballpoint pens, inkwash and sometimes watercolours or gouache. I prefer creating my work that way. To me keeping that 'hand-crafted' style and aesthetic has its charm when bringing it to life through Augmented Reality. I think it's really important to demonstrate the importance of other traditional mediums, techniques and disciplines when working with immersive technologies as they're still very valid in creating engaging content. I see a lot of AR experiences that have a lack of charm and feel somewhat 'gamey' which is why I want to continue integrating more raw-looking aesthetics in this area.

Augmented Representation & Implementation

Once the art is planned and created, I then scan it and start splicing it up digitally. From the preserved textures and linework I then create a 3D representation of the artwork through 3D modeling and animation software called Blender. It's one of many 3D authoring tools out there. It not only packs a significant number of features, it's also free which makes it ideal for beginners and students.

Once the 3D representation of the work is created it's then imported into a game engine called Unity3D where I implement the AR functionality. Unity 3D is a widely used game engine. It's one of many engines out there but what makes it great is its support to deploy to all manners of devices. It also has a large community base behind it should you need help.

How long does a typical project take? Do you work by yourself or with a team? How long do you work?

On average an Augmented Artwork may take anywhere from 3 to 4 weeks to create, which includes the Art, Animation, Modeling and AR Implementation. When I first started out it'd take much longer but overtime I've streamlined my own processes to get things done faster.

My alias Marc-O-Matic tends to be mistaken as a team but I'm actually just one person. I much prefer being able to create and direct everything myself.

What is your advice to a budding artist who is interested in learning tech to produce art?

From my experience, don't confine yourself to one specific medium and try practising a different skill or technique everyday. As artists we should be constantly evolving not just our technical skills but also our thought processes to push the boundaries on what's achievable. 'How can I tell my story in a more engaging and interesting manner?' 'What can I create if I combine these different skill sets together?" "How do I bring an end to systematic racism and white privilege?' etc. The more knowledge you have the greater you can push the boundaries of your creative expression.

How do you support yourself financially? As in do you also sell your art pieces or make art for clients etc.? Where do you sell stuff, and meet people?

I work under my own Artist Entity 'Marc-O-Matic.' I much prefer working independently than within larger studios or agencies. I juggle a balance between commercial work and personal projects. There's generally a high demand for Augmented/Virtual Reality experiences and I'm really lucky and grateful that clients want content in my particular style. I work with a variety of organisations generally consulting, providing creative direction and in most cases building the experiences too.

Aside from the commercial work, I'm currently touring my own Augmented Reality Art/Storytelling collection called 'Moving Marvels' where audiences get to see my illustrated works come to life right in front of them. It's also how I sell various limited edition prints of my augmented artworks. My collection is exhibited at tech/innovation conferences, symposiums, galleries and even at universities/educational institutes. It's a great way to make connections and demonstrate what immersive technologies can do in a creative capacity.

No interview would be complete without a discussion on AI and machine learning. As a technophile artist, are you excited about merging AI and art in new ways? What are some of your hopes and fears about this new tech? Do you plan to try machine learning anytime soon?

It’s like a double edged sword with + 5 to Creativity. Technology can enhance our creative abilities and processes in a number of ways. At the same time it can also make some of us lazy because we can become so reliant on it.
Like any new technology that aims to assist in the creative process, there’s always the fear that technology will make creatives lazy or will replace professions altogether. In some ways technology has done this but from saying that it has also created new job opportunities and abilities for artists.

In areas like character animation for example the assistance of machine learning algorithms means creatives can worry less about the laborious physical processes of rigging and complex animation and focus more on the storytelling.

For example, we can significantly reduce production times in the areas of facial rigging through facial recognition. Through learning the behaviour and structure of your own face, machine learning algorithms can map your facial movements directly to a 3D character’s face. By simply recording your own facial movements and gestures, you’ve created an entire impression map for your 3D character to use. What’s also crazy is, on top of that, by running a voice over track on that 3D character, you can also train it to play it out to sync with the voice AS well as have the entire face emotionally react based on the tone of the audio. It’s game-changing but also terrifying stuff as this sort of technology can be used to create highly realistic fake impressions of real people.

As also an animator, I’ve started experimenting with this technology for my own VR animations. What would take me animator hours or even days to animate a talking 3D character can now take mere minutes.

Author Bio

interview-with-marco-matic-marco-ryan-augmented-reality-artist-img-0

Marc-O-Matic is the moniker of Marco Matic Ryan. He is a multidisciplinary Artist, Animator, Director, Storyteller and Technologist working across Augmented and Virtual Reality technologies. He is based in Victoria, Australia.

https://player.vimeo.com/video/125218939

Unity plugins for augmented reality application development.

Google ARCore is pushing immersive computing forward.

There’s another player in the advertising game: augmented reality.