Anybody who’s ever worked in a client services industry knows this feeling: you're on task, doing what you’re supposed to do, confident in the creative work you’re creating, and then—BAM! An obstacle hits you smack in the face.

Everybody handles challenges in their own ways. Our way? Give it another thought. Ask yourself: Can I be flexible? Can I think about this project in a whole new way? Yallie says you can.

, designed and executed by Director of 3D Sehzat Oner (sheh-ZAHT OH-ner). Sehzat created a 3D model of the beloved Yallie mascot (brainchild of Creative Director Mike Foster) in Blender, then transported it into Houdini to magically transform into sand, collapse, and fly away using realistic physics models. Rendered with Karma. Score inspired by The Chordettes's “Mr. Sandman” and created by Sehzat on his vintage Jupiter 6.

Anybody who’s ever worked in a client services industry knows this feeling: you're on task, doing what you’re supposed to do, confident in the creative work you’re creating, and then—BAM! An obstacle hits you smack in the face.

Everybody handles challenges in their own ways. Our way? Give it another thought. Ask yourself: Can I be flexible? Can I think about this project in a whole new way? Yallie says you can.

How?

This is the first in a series of Lessons of Yallie, designed and executed by Director of 3D Sehzat Oner (sheh-ZAHT OH-ner). Sehzat created a 3D model of the beloved Yallie mascot (brainchild of Creative Director Mike Foster) in Blender, then transported it into Houdini to magically transform into sand, collapse, and fly away using realistic physics models. Rendered with Karma. Score inspired by The Chordettes's “Mr. Sandman” and created by Sehzat on his vintage Jupiter 6.

Getting stuck is the stuff dreams are made of.

Lessons of Yallie

Human red blood cells are about three ten-thousandths of an inch across (no big deal!), and they're coursing through our veins at a relatively speedy three miles per hour, crashing into each other like bike messengers delivering oxygen.

Thanks to modern technology, you can now float down a peaceful river of blood cells right here in your browser. It's like teeny tiny red river rafting, without the added risks of 

 being shrunk down to that size (stepped on by cats, eaten by an earwig, etc.)

Creative Technologist Christian Kastner built this demo using the ThreeJS libraries React Three Fiber and Drei. He took it one step further by using the Rapier physics engine and linear algebra to demonstrate the cells colliding, clotting, and jostling downstream, as well as letting you interact with them. Director of 3D Animation Sehzat Oner created the blood cell model in Blender.

Displacing the vessels with the users' mouse was a particularly difficult challenge. Your mouse moves an invisible object that blocks the cells' path. Both this object and the camera through which you view the experience, like Spiritualized, are floating 

 in space. How do you tell the object where to go, while getting it and the camera to look at each other? Easy:

: a particularly confusing mathematical way of computing transformations in 3-dimensions. After days of testing, we found an elegant solution in hiding another invisible object in the scene and using its built-in functionality to transform the relatively straightforward Euler rotations into quaternions. And voila, your mouse now has physics, and the experience an added dose of realism. It's not always easy being real, but it's the only way for us to be.

Human red blood cells are about three ten-thousandths of an inch across (no big deal!), and they're coursing through our veins at a relatively speedy three miles per hour, crashing into each other like bike messengers delivering oxygen.

Thanks to modern technology, you can now float down a peaceful river of blood cells right here in your browser. It's like teeny tiny red river rafting, without the added risks of actually being shrunk down to that size (stepped on by cats, eaten by an earwig, etc.)

How?

Creative Technologist Christian Kastner built this demo using the ThreeJS libraries React Three Fiber and Drei. He took it one step further by using the Rapier physics engine and linear algebra to demonstrate the cells colliding, clotting, and jostling downstream, as well as letting you interact with them. Director of 3D Animation Sehzat Oner created the blood cell model in Blender.

Displacing the vessels with the users' mouse was a particularly difficult challenge. Your mouse moves an invisible object that blocks the cells' path. Both this object and the camera through which you view the experience, like Spiritualized, are floating somewhere in space. How do you tell the object where to go, while getting it and the camera to look at each other? Easy:

That's right, it's quaternions: a particularly confusing mathematical way of computing transformations in 3-dimensions. After days of testing, we found an elegant solution in hiding another invisible object in the scene and using its built-in functionality to transform the relatively straightforward Euler rotations into quaternions. And voila, your mouse now has physics, and the experience an added dose of realism. It's not always easy being real, but it's the only way for us to be.

Shrink yourself down to a microscopic level and experience 

Blood Vessel Voyage

Our favorite mascot has taken to the Augmented Reality space and throwing down everything he has. And to boot, it all works in a browser! With platforms like 8th Wall, Zapworks, or Onirix, webAR has made it incredibly easy for users to jump into a branded AR experience, snap a pic with a custom face filter, or make your advertisement images come alive.

For this lab, we took to using 8th wall as our webAR platform and A-frame as the coding framework and a then popped in a rigged version of Yallie into Adobe's mixamo to add some funky animations to him.

This platform is probably the most varied of all the tools at offer. We have options like hand-tracking, face-tracking, or image-tracking to fix 3D models to these real world objects. There's also an ability to register the sky and have massive 3D objects flying across the sky line. And finally 8th Wall boasts the best world tracking (Simultaneous Localization and Mapping or SLAM) in the webAR world. So whole virtual worlds are interact-able right through your phone's browser.

A-frame as a framework makes quick work of a project like this as we just make a scene, a camera (that's controlled by the motion of your phone), load our 3D model of Yallie with all his dance moves. And finally we place an invisible box around our scene so that when our user touches the screen, we can find where on the box this action happened and place yallie right there. The really nice fact about this framework is that it looks startingly like regular HTML and in fact we place our A-frame tags right in the body of our HTML!

But being the crafty programmers that we are, we can develop combine webAR with much more technical frameworks (not for the faint of heart) like Threejs or React Three Fiber. This allows us to have full control over the materials, functionality, and interactivity of our 3D scenes to build pretty much whatever we'd like!

Finally, we harnessed the power of Adobe's free 3D animation tool to rig our big blue guy to leave it all on the dance floor. But it's not just dancing that we can do, there's also a whole host of other prebuilt animations that we can incorporate. Be it running, jumping, fighting, the library is vast and enables us to quickly add new animations to any rigged humanoid model that we have! The opportunities are endless.

That about cuts it. With the mainstream adoption of VR/AR still emerging, it's more important than ever to see the possibilities that webAR has. Just like websites in the early 2000s, AR is seemingly that next evolution of digital branding. And as the technology gets better and better, we might just see the physical world becoming less realized without the digital.

Our favorite mascot has taken to the Augmented Reality space and throwing down everything he has. And to boot, it all works in a browser! With platforms like 8th Wall, Zapworks, or Onirix, webAR has made it incredibly easy for users to jump into a branded AR experience, snap a pic with a custom face filter, or make your advertisement images come alive.

Get Down on it

For this lab, we took to using 8th wall as our webAR platform and A-frame as the coding framework and a then popped in a rigged version of Yallie into Adobe's mixamo to add some funky animations to him.

8th Wall

This platform is probably the most varied of all the tools at offer. We have options like hand-tracking, face-tracking, or image-tracking to fix 3D models to these real world objects. There's also an ability to register the sky and have massive 3D objects flying across the sky line. And finally 8th Wall boasts the best world tracking (Simultaneous Localization and Mapping or SLAM) in the webAR world. So whole virtual worlds are interact-able right through your phone's browser.



A-frame

A-frame as a framework makes quick work of a project like this as we just make a scene, a camera (that's controlled by the motion of your phone), load our 3D model of Yallie with all his dance moves. And finally we place an invisible box around our scene so that when our user touches the screen, we can find where on the box this action happened and place yallie right there. The really nice fact about this framework is that it looks startingly like regular HTML and in fact we place our A-frame tags right in the body of our HTML!

But being the crafty programmers that we are, we can develop combine webAR with much more technical frameworks (not for the faint of heart) like Threejs or React Three Fiber. This allows us to have full control over the materials, functionality, and interactivity of our 3D scenes to build pretty much whatever we'd like!

Mixamo

Finally, we harnessed the power of Adobe's free 3D animation tool to rig our big blue guy to leave it all on the dance floor. But it's not just dancing that we can do, there's also a whole host of other prebuilt animations that we can incorporate. Be it running, jumping, fighting, the library is vast and enables us to quickly add new animations to any rigged humanoid model that we have! The opportunities are endless.

Finishing up

That about cuts it. With the mainstream adoption of VR/AR still emerging, it's more important than ever to see the possibilities that webAR has. Just like websites in the early 2000s, AR is seemingly that next evolution of digital branding. And as the technology gets better and better, we might just see the physical world becoming less realized without the digital.



Scan our QR code here! 



An interactive AR experience with our beloved Yallie, all in your browser

Yallie Likes to Move it Move it

Look, y'all have been there. You're in your skyscraper working on your slide deck for the big oat milk pitch when—gasp!—suddenly, outside your window a giant blue biped with sympathetic eyes appears. They're traipsing through town. They're knocking over buildings. They're shouting their own name. 

Yallie's a larger-than-life figure around here, so what better way to celebrate them than by making a mini game where you can play dominoes with a cityscape? 

Creative Technologist Christian Kastner conceived of and built this demo using the React Three Fiber and Drei libraries for ThreeJS, with the Cannon physics engine. Director of 3D Animation Sehzat Oner sculpted and rigged the model in Blender. Eponymous roars brought to you by Director of Creative Technology Michael Fanuzzi.

Look, y'all have been there. You're in your skyscraper working on your slide deck for the big oat milk pitch when—gasp!—suddenly, outside your window a giant blue biped with sympathetic eyes appears. They're traipsing through town. They're knocking over buildings. They're shouting their own name. It must be that darn Yallie again!

Yallie's a larger-than-life figure around here, so what better way to celebrate them than by making a mini game where you can play dominoes with a cityscape? Use your arrow keys to explore.

How?

Creative Technologist Christian Kastner conceived of and built this demo using the React Three Fiber and Drei libraries for ThreeJS, with the Cannon physics engine. Director of 3D Animation Sehzat Oner sculpted and rigged the model in Blender. Eponymous roars brought to you by Director of Creative Technology Michael Fanuzzi.

A surprisingly satisfying interactive 3D game experience.

Yallzilla

Part of moving towards a more engaging experience on the web means experimenting with newer technologies. And—for all of us at 

—that means taking advantage of the leaps and bounds of growth in modern browser capabilities that can offer an extra "wow" factor to user interaction. For Front-End Engineer Timothy Merritt, the creation of these "Alchemicals" was rooted in a desire to become more familiar with powerful 3D technologies, while offering a playful example of what's possible for immersive UI/UX.

At its core, most 3D technology is really just a fancy magic trick. The way our eyes perceive motion and depth can be easily influenced (motion pictures have been succeeding at it for over a century), but we nonetheless delight at suspending our disbelief and engaging with the illusions. For the digital realm, the same principle applies—there's just a lot more math involved under the hood.

 what you're seeing is more traditional two-dimensional "background" text that's given depth in conjunction with 3D elements floating "above" it. The camera view tilts and pans according to where the cursor is located (move the cursor to the right side of the screen and the "camera" pans right, etc), and each of the colorful blob objects can be clicked and dragged around to squish into one another. The goal is to forget the illusion while you're enjoying pushing blobs together and pulling them apart, the way a child does when they play with their toys.

A basic physics engine is at play here as well, creating both a weak gravitational pull towards the center of the screen for the blobs, as well as a defined collision logic for when they bump into one another. There's two states to this environment:

— If no user interaction (e.g., clicking and dragging) is taking place, the blobs will naturally move according to the rules of the given gravity settings.

— Otherwise the blob will "follow" the cursor when being dragged around. This is what creates the illusion that you're grabbing and moving the blobs in a three-dimensional space with your cursor.

Each blob has a three dimensional "geometry"—that is, an underlying wireframe skeleton that tells the browser to show it as a sphere—along with a secondary "gooey" layer atop that geometry, which is what interacts when one or more of the blobs collide. It's the strength (or lack thereof) of that gooey layer that allows for the blobs to appear as though they're melting into one another once they are close enough in proximity.

Additionally, each blob is rendered slightly transparent, with a particular color assigned and various material effects applied to the blob object: chromatic aberrations (slight iridescent refraction), temporal distortion (a "wobbly" effect), and numerous other properties including thickness, roughness, and "metalness".

While integrating 3D into web interfaces can be deeply engaging, there are also important caveats to consider. Traditional interface elements have had a long history of revision and interaction with broad numbers of users, and conversely, users are accustomed to know how to use buttons, hamburger menus, sliders, etc., when using a website or app. 3D interactions can sometimes fall outside of or clash against that paradigm, and including them in UI/UX workflows often means some sort of trade-off will occur.

3D elements are typically not accessible, or at best are very difficult to make adequately accessible for users who might have visual or motion-related impairments. They also demand a greater degree of energy and processing power from a user's device, and that device's GPU capabilities can dictate (or throttle) how 3D elements will perform.

As a result, in practice, it's best to try and balance any 3D heavy elements with traditional interactive elements to give users an anchor they're familiar with (you wouldn't want a 3D-heavy form, for example), or employ 3D in a way that won't affect a user's experience in using the broader functionality of a web application. These Alchemicals are thus mostly a fun little interactive element that won't impede a user's journey through rest of the given site or app.

 (and its subsequent libraries, Drei, Fiber, and Rapier) for 3D rendering within a 

Part of moving towards a more engaging experience on the web means experimenting with newer technologies. And—for all of us at y'all—that means taking advantage of the leaps and bounds of growth in modern browser capabilities that can offer an extra "wow" factor to user interaction. For Front-End Engineer Timothy Merritt, the creation of these "Alchemicals" was rooted in a desire to become more familiar with powerful 3D technologies, while offering a playful example of what's possible for immersive UI/UX.

3D and depth creation

At its core, most 3D technology is really just a fancy magic trick. The way our eyes perceive motion and depth can be easily influenced (motion pictures have been succeeding at it for over a century), but we nonetheless delight at suspending our disbelief and engaging with the illusions. For the digital realm, the same principle applies—there's just a lot more math involved under the hood.

In the case of the Alchemicals, what you're seeing is more traditional two-dimensional "background" text that's given depth in conjunction with 3D elements floating "above" it. The camera view tilts and pans according to where the cursor is located (move the cursor to the right side of the screen and the "camera" pans right, etc), and each of the colorful blob objects can be clicked and dragged around to squish into one another. The goal is to forget the illusion while you're enjoying pushing blobs together and pulling them apart, the way a child does when they play with their toys.

Physics

A basic physics engine is at play here as well, creating both a weak gravitational pull towards the center of the screen for the blobs, as well as a defined collision logic for when they bump into one another. There's two states to this environment:

— If no user interaction (e.g., clicking and dragging) is taking place, the blobs will naturally move according to the rules of the given gravity settings.

— Otherwise the blob will "follow" the cursor when being dragged around. This is what creates the illusion that you're grabbing and moving the blobs in a three-dimensional space with your cursor.



Geometry

Each blob has a three dimensional "geometry"—that is, an underlying wireframe skeleton that tells the browser to show it as a sphere—along with a secondary "gooey" layer atop that geometry, which is what interacts when one or more of the blobs collide. It's the strength (or lack thereof) of that gooey layer that allows for the blobs to appear as though they're melting into one another once they are close enough in proximity.

Additionally, each blob is rendered slightly transparent, with a particular color assigned and various material effects applied to the blob object: chromatic aberrations (slight iridescent refraction), temporal distortion (a "wobbly" effect), and numerous other properties including thickness, roughness, and "metalness".

Knowing which rules to break

While integrating 3D into web interfaces can be deeply engaging, there are also important caveats to consider. Traditional interface elements have had a long history of revision and interaction with broad numbers of users, and conversely, users are accustomed to know how to use buttons, hamburger menus, sliders, etc., when using a website or app. 3D interactions can sometimes fall outside of or clash against that paradigm, and including them in UI/UX workflows often means some sort of trade-off will occur.

3D elements are typically not accessible, or at best are very difficult to make adequately accessible for users who might have visual or motion-related impairments. They also demand a greater degree of energy and processing power from a user's device, and that device's GPU capabilities can dictate (or throttle) how 3D elements will perform.

As a result, in practice, it's best to try and balance any 3D heavy elements with traditional interactive elements to give users an anchor they're familiar with (you wouldn't want a 3D-heavy form, for example), or employ 3D in a way that won't affect a user's experience in using the broader functionality of a web application. These Alchemicals are thus mostly a fun little interactive element that won't impede a user's journey through rest of the given site or app.

Magic Technologies used

The Alchemicals use React Spring for natural physics reactions and React Three (and its subsequent libraries, Drei, Fiber, and Rapier) for 3D rendering within a Vite.js deployment, along with twin.macro for some basic styling.

Alchemicals

We know that while 3D environments can be magical and mesmerizing, creating a convincing one can also be tedious and time consuming. So, when our lead animator Sehzat at got the spark to create his next immersive story, he figured out a way to be both accurate AND agile. Experimenting with a 3D growth process within Houdini he has begun to create a virtual world that blossoms into reality right before the viewers’ eyes. Using a combination of Midjourney for storyboard creation and photoshop generative AI for the backplate, he was able to develop a realistic look and feel, focusing more on color and procedurally generated materials vs texture mapping.


Overall, this AI boost is allowing more time to focus on the bigger picture—growing the vision and the story--and bringing it to life much more quickly. See the sneak peek below

We know that while 3D environments can be magical and mesmerizing, creating a convincing one can also be tedious and time consuming. So, when our lead animator Sehzat at got the spark to create his next immersive story, he figured out a way to be both accurate AND agile. Experimenting with a 3D growth process within Houdini he has begun to create a virtual world that blossoms into reality right before the viewers’ eyes. Using a combination of Midjourney for storyboard creation and photoshop generative AI for the backplate, he was able to develop a realistic look and feel, focusing more on color and procedurally generated materials vs texture mapping.


Overall, this AI boost is allowing more time to focus on the bigger picture—growing the vision and the story--and bringing it to life much more quickly. See the sneak peek below

At y’all, when we get the spark to create, we don’t like to wait.

Notes of Forest Floor

Recently we were working on a new project with a prominent 3D element and played around with a few different interactions. Although this one didn't make the cut, we thought it was an interesting enough interaction that you don't see much in the world of 3D on the web. What better way to show case than in a lab!

As you can see in the demo above, your mouse cursor will cause the 3D object to grow and distort from its original position. The technical term for this is 

. 3D models are composed of small points connected together called vertices. Moving these groups of points in response to the mouse allow us to change and distort the object and create a fun, dynamic interaction.

However, figuring out how close or far away our mouse is to an individual vertex's position requires... math. We must translate our mouse's position on the computer screen into positions in the virtual 3D space. This is complicated. Or we could do the reverse- translate the 3D space onto our screen. Luckily, the 3D web pipeline does the latter for us, so we can simply intercept the point position in this pipeline and operate from there.

Next, we can tweak the strength of our effect, and how it becomes less powerful as our mouse moves farther and farther away from the object, and more powerful as it moves closer. Here, we are morphing the shape, but the use cases for an interaction like this are endless: we can distort the object, have the mouse "paint" it with a color, make it transparent... the list goes on!

Recently we were working on a new project with a prominent 3D element and played around with a few different interactions. Although this one didn't make the cut, we thought it was an interesting enough interaction that you don't see much in the world of 3D on the web. What better way to show case than in a lab!

As you can see in the demo above, your mouse cursor will cause the 3D object to grow and distort from its original position. The technical term for this is vertex displacement. 3D models are composed of small points connected together called vertices. Moving these groups of points in response to the mouse allow us to change and distort the object and create a fun, dynamic interaction.

The challenge

However, figuring out how close or far away our mouse is to an individual vertex's position requires... math. We must translate our mouse's position on the computer screen into positions in the virtual 3D space. This is complicated. Or we could do the reverse- translate the 3D space onto our screen. Luckily, the 3D web pipeline does the latter for us, so we can simply intercept the point position in this pipeline and operate from there.

Next, we can tweak the strength of our effect, and how it becomes less powerful as our mouse moves farther and farther away from the object, and more powerful as it moves closer. Here, we are morphing the shape, but the use cases for an interaction like this are endless: we can distort the object, have the mouse "paint" it with a color, make it transparent... the list goes on!

A vivacious verse about the vicissitudes of a very simple vertex

V for Vertex

, we're always looking for new ways to create subtle — yet, awe-inspiring — interactions with technology and art. 

For a recent conference that took place in Chicago, we used Unreal Engine to create a kidney-shaped homage to the city's iconic "

." We placed two cameras within a giant video display, capturing a stereo image of the attendees and projecting that onto a reflective kidney within a cityscape. Attendees even took selfies in front of it, just like the tourists at Millennium Park! This was fun, but it was only the start. We wanted to take the idea a step further and make it 

interactive. But we didn't want to complicate things and clutter the booth with sensors, which also give away how the magic works.

How do you make a 24' screen interactive without a ton of sensors or extra hardware?

To do this, we needed real-time person detection and tracking. Prior to the AI revolution, this would have been done using arrays of motion detectors, like the kind you might see attached to flood lights on a house, that rely on ultrasonic waves or lasers. They're big, so you have to find someplace to put them. They're not so accurate. And having additional components makes setup, development and testing that much more complicated. Not to mention the cost. Being we already have cameras going into a computer, what if the computer could "see" the people and let us know where they were, like motion detectors would?

This is a field of research called "computer vision," where AI and algorithms try to figure out what's going on using a video or image. This is not easy! Scouring the internet, we found a few suitable machine learning algorithms but one stood out. The model goes by the name of 

 🥁🐍) and its combination of speed and robustness stood head and shoulders above the other techniques we found. Alongside its detection of human beings, it could even detect a myriad of other objects — including phones, remotes, cars, chairs, you name it!

Our initial prototypes ran video from our cameras' 

 streams into YOLO, which processed each frame, and then either calculated how large the person was in the frame, using the bounding box area (above, in red), or figured out the position of the person by getting the center of the bounding box. This allowed us to distinguish when a person stopped in front of the exhibit, versus when they just walked by. Knowing their position allowed us to show them content wherever they were along the 24-foot long screen.

We refined our technique into a standalone python app, which would turn video into a set of anonymous numbers, and send this data into Unreal via 

. Thereby allowing us to control any 3D elements with people's movement! And because the video itself is not saved or further transmitted, it's also both private and super fast as well:

Where the virtual world meets the real world... and not in a weird sci-fi way.

What used to be reserved for specialists and researchers is now our state-of-the-art. AI is replacing hardware, resulting in greater creative possibilities 

 lets us leverage some of the best technology of the video game industry, with which we can create fast, beautiful interactive worlds. When you put these two together with a bit of creativity, we believe it's the next step in making awe-inspiring interactive experiences for physical spaces.

Here at y'all agency, we're always looking for new ways to create subtle — yet, awe-inspiring — interactions with technology and art. 

For a recent conference that took place in Chicago, we used Unreal Engine to create a kidney-shaped homage to the city's iconic "Bean." We placed two cameras within a giant video display, capturing a stereo image of the attendees and projecting that onto a reflective kidney within a cityscape. Attendees even took selfies in front of it, just like the tourists at Millennium Park! This was fun, but it was only the start. We wanted to take the idea a step further and make it even more interactive. But we didn't want to complicate things and clutter the booth with sensors, which also give away how the magic works.

How do you make a 24' screen interactive without a ton of sensors or extra hardware?

To do this, we needed real-time person detection and tracking. Prior to the AI revolution, this would have been done using arrays of motion detectors, like the kind you might see attached to flood lights on a house, that rely on ultrasonic waves or lasers. They're big, so you have to find someplace to put them. They're not so accurate. And having additional components makes setup, development and testing that much more complicated. Not to mention the cost. Being we already have cameras going into a computer, what if the computer could "see" the people and let us know where they were, like motion detectors would?

This is a field of research called "computer vision," where AI and algorithms try to figure out what's going on using a video or image. This is not easy! Scouring the internet, we found a few suitable machine learning algorithms but one stood out. The model goes by the name of YOLO (You Only Look Once 🥁🐍) and its combination of speed and robustness stood head and shoulders above the other techniques we found. Alongside its detection of human beings, it could even detect a myriad of other objects — including phones, remotes, cars, chairs, you name it!

Our initial prototypes ran video from our cameras' NDI streams into YOLO, which processed each frame, and then either calculated how large the person was in the frame, using the bounding box area (above, in red), or figured out the position of the person by getting the center of the bounding box. This allowed us to distinguish when a person stopped in front of the exhibit, versus when they just walked by. Knowing their position allowed us to show them content wherever they were along the 24-foot long screen.

We refined our technique into a standalone python app, which would turn video into a set of anonymous numbers, and send this data into Unreal via OSC. Thereby allowing us to control any 3D elements with people's movement! And because the video itself is not saved or further transmitted, it's also both private and super fast as well:

Where the virtual world meets the real world... and not in a weird sci-fi way.

What used to be reserved for specialists and researchers is now our state-of-the-art. AI is replacing hardware, resulting in greater creative possibilities and more efficient installations. Unreal Engine lets us leverage some of the best technology of the video game industry, with which we can create fast, beautiful interactive worlds. When you put these two together with a bit of creativity, we believe it's the next step in making awe-inspiring interactive experiences for physical spaces.



🔥YＯLＯ🔥 in a Different Way

, both in our words and in our work. So it's only fitting that the hero of our homepage should literally 

 this 🥁. So we built, from scratch, a fun interactive 3d composition that combines a pair of modern techniques in a unique way, to create not just a clear, glass-like orb, but one that morphs and squiggles as you touch it with your mouse. Also just like us? 

This blobby gel-y goodness is achieved through a 3D technique known as 

The reflections and distortions caused by the blob, similar to what you'd see looking through an old windowpane or a glass bottle, are known as refraction, which are achieved with a 

. But why, you may ask, did we put these two together? Well, firstly because we believe in learning through experimentation here at y'all labs. But secondly, because we can.

 the below content is technical, but we will try to make it understandable. You can leave that graphing calculator at home.

Look at some object, any object, in the distance. Now imagine an arrow beaming from your eyes to that object. That arrow is a "ray."

You likely only have two eyes, but now imagine that every pixel on your monitor is an "eye" of sorts, looking straight behind it into a 3d scene.  Each of those eyes beams out (emits) an arrow (ray) that travels (marches) farther and farther on until it collides with an object, and tells us what the thing it hit looks like. Your monitor puts those all together in a grid to form the image you see above.

So for each pixel in our image, we'll have one ray.

along our scene until they hit upon an object. How does it known when it's hit an object? Unlike a typical 3d scene where we place objects in a field and can say "there is a box exactly 3 feet from the camera and a 1 foot to the left," with this technique we have to shape and place our objects using math. 😱

To do that we use something called a Signed Distance Field (SDF). This is a mathematical function that describes the shape of a 3d object (box, sphere, etc.) It returns a positive value if the ray lands outside of an object or negative if it lands inside. So returning to our monitor, we send out a ray from each pixel, and if the SDF for that pixel returns a value less than or equal to 0, then bingo, we have a collision, and we know that pixel will show 

, which helped us in developing our own unique shapes.)

Once the raymarcher tells us we've hit something, we know that the pixel should show our object. As a simple example, let's suppose we're using an SDF that generates a sphere, where a hit means we shade that pixel white and all else is black. Then we'd have something like this:


Just a boring circle with zero depth to it. To add dimensionality, we need to add a lighting model to it, to simulate how light interacts with a sphere IRL.

There are many different lighting models under the sun that can help us bring the sphere to life. But one of the easier models that we can use is an "image based" approach.

Image Based Lighting (IBL) is a 3D graphics technique that helps make scenes look realistic by using images to capture the surrounding environment's lighting and colors. Think of it like bringing a shiny glass ball into a room with colorful walls; the ball will reflect the colors around it, blending naturally with its surroundings. In IBL, we make those "walls" out of images, so when an object like our sphere is placed in this digital environment, it interacts with the light and colors from these images, creating realistic reflections, shadows, and illumination.

This series of images is called a cubemap and you can think of it like a six sided cube that surrounds our scene:


When a ray hits our object and reflects off it, we can use that resulting direction as a value hitting our surrounding cubemap. A completely reflective object like a mirror will be a clear or blurred copy of the surrounding cubemap depending on the "roughness" of the surface like so:

We use a programming language called GLSL to help create these reflections. First, you find out how light interacts with the object's surface at the point it hits, using a "normal," which is a line standing straight up from the surface at that point, much like a stick standing upright on a flat table. By knowing this normal and the direction from which the light comes (the "incident ray"), you can figure out how the light will bounce off or pass through the object, creating a glass-like look and distorting the view of objects around it.

However like mentioned before, and as you've no doubt noticed in real life, glass is not completely reflective. Most light will pass through the surface (refract), but some will reflect back to you based on the angle of the light (incident ray). This is what's known as the 

. Very simply, it is the observation that amount of reflection you see on a surface depends on the viewing angle.

Whereas in real life the Fresnel is dependent upon types of materials and light sources, in computer graphics, we can customize this to our desired effect. Here, we wanted the sides of our glass to reflect our cubemap more strongly while the interior of the sphere will refract through the object, resulting in a defined edge and clear-ish center:

In the physical world, materials have an Index of Refraction (IOR), which measures the behavior of light as it passes from objects of differing densities. Luckily, the language we are building this in (GLSL), has a built-in refraction function that does the heavy lifting for us given a virtual IOR value. For glass, the IOR is around 1.52. That gives us this result:

Our background text is essentially static. Thus, we can treat the background text as an image and use the same idea as our IBL cubemap as described previously. Notice how the image is actually flipped, just like a real world glass sphere!

The one thing that is missing now is the chromatic aberration that happens with glass, where the different wavelengths of light are refracted differently creating an amazing dance of color. While with modern clear glass this is so subtle as to be unnoticeable, you can see this effect more strongly on older leaded glass windows, which act like a kind of prism, separating the different wavelengths of light.

We can achieve the same effect by simply offsetting the IOR values slightly for each color channel of red, green and blue. This means calculating a different refraction direction for each color and adding them back together to create our image like so:

In it's basic form, this is how we achieved the look of our site's hero. Through the magic of raymarching and a little technical shading work we were able to create a performant and interactive experience for the web. We customized the glass IOR, fresnel, and chromatic aberration values to make something more fantastic than realistic, but we hope these comparisons to real-world phenomena have given you an idea about what went on behind the scenes here. 

For more detail on these concepts (and you thought this was technical!) check out these links, which helped us in developing this article:

Here at y'all, we are built differently, both in our words and in our work. So it's only fitting that the hero of our homepage should literally reflect this 🥁. So we built, from scratch, a fun interactive 3d composition that combines a pair of modern techniques in a unique way, to create not just a clear, glass-like orb, but one that morphs and squiggles as you touch it with your mouse. Also just like us? Ahem.

This blobby gel-y goodness is achieved through a 3D technique known as raymarching. The reflections and distortions caused by the blob, similar to what you'd see looking through an old windowpane or a glass bottle, are known as refraction, which are achieved with a shader. But why, you may ask, did we put these two together? Well, firstly because we believe in learning through experimentation here at y'all labs. But secondly, because we can.

Warning: the below content is technical, but we will try to make it understandable. You can leave that graphing calculator at home.

Who is Ray and why does he March

Look at some object, any object, in the distance. Now imagine an arrow beaming from your eyes to that object. That arrow is a "ray."

You likely only have two eyes, but now imagine that every pixel on your monitor is an "eye" of sorts, looking straight behind it into a 3d scene.  Each of those eyes beams out (emits) an arrow (ray) that travels (marches) farther and farther on until it collides with an object, and tells us what the thing it hit looks like. Your monitor puts those all together in a grid to form the image you see above.

So for each pixel in our image, we'll have one ray.

The rays will march along our scene until they hit upon an object. How does it known when it's hit an object? Unlike a typical 3d scene where we place objects in a field and can say "there is a box exactly 3 feet from the camera and a 1 foot to the left," with this technique we have to shape and place our objects using math. 😱

To do that we use something called a Signed Distance Field (SDF). This is a mathematical function that describes the shape of a 3d object (box, sphere, etc.) It returns a positive value if the ray lands outside of an object or negative if it lands inside. So returning to our monitor, we send out a ray from each pixel, and if the SDF for that pixel returns a value less than or equal to 0, then bingo, we have a collision, and we know that pixel will show something. But what?

(Shout out to this great article by Inigo Quilez, which helped us in developing our own unique shapes.)

Shading and Refraction

Once the raymarcher tells us we've hit something, we know that the pixel should show our object. As a simple example, let's suppose we're using an SDF that generates a sphere, where a hit means we shade that pixel white and all else is black. Then we'd have something like this:


Just a boring circle with zero depth to it. To add dimensionality, we need to add a lighting model to it, to simulate how light interacts with a sphere IRL.

Image Based Lighting

There are many different lighting models under the sun that can help us bring the sphere to life. But one of the easier models that we can use is an "image based" approach.

Image Based Lighting (IBL) is a 3D graphics technique that helps make scenes look realistic by using images to capture the surrounding environment's lighting and colors. Think of it like bringing a shiny glass ball into a room with colorful walls; the ball will reflect the colors around it, blending naturally with its surroundings. In IBL, we make those "walls" out of images, so when an object like our sphere is placed in this digital environment, it interacts with the light and colors from these images, creating realistic reflections, shadows, and illumination.

This series of images is called a cubemap and you can think of it like a six sided cube that surrounds our scene:


When a ray hits our object and reflects off it, we can use that resulting direction as a value hitting our surrounding cubemap. A completely reflective object like a mirror will be a clear or blurred copy of the surrounding cubemap depending on the "roughness" of the surface like so:

We use a programming language called GLSL to help create these reflections. First, you find out how light interacts with the object's surface at the point it hits, using a "normal," which is a line standing straight up from the surface at that point, much like a stick standing upright on a flat table. By knowing this normal and the direction from which the light comes (the "incident ray"), you can figure out how the light will bounce off or pass through the object, creating a glass-like look and distorting the view of objects around it.

Refraction

However like mentioned before, and as you've no doubt noticed in real life, glass is not completely reflective. Most light will pass through the surface (refract), but some will reflect back to you based on the angle of the light (incident ray). This is what's known as the Fresnel effect. Very simply, it is the observation that amount of reflection you see on a surface depends on the viewing angle.

Whereas in real life the Fresnel is dependent upon types of materials and light sources, in computer graphics, we can customize this to our desired effect. Here, we wanted the sides of our glass to reflect our cubemap more strongly while the interior of the sphere will refract through the object, resulting in a defined edge and clear-ish center:

In the physical world, materials have an Index of Refraction (IOR), which measures the behavior of light as it passes from objects of differing densities. Luckily, the language we are building this in (GLSL), has a built-in refraction function that does the heavy lifting for us given a virtual IOR value. For glass, the IOR is around 1.52. That gives us this result:

Our background text is essentially static. Thus, we can treat the background text as an image and use the same idea as our IBL cubemap as described previously. Notice how the image is actually flipped, just like a real world glass sphere!

Chromatic aberration

The one thing that is missing now is the chromatic aberration that happens with glass, where the different wavelengths of light are refracted differently creating an amazing dance of color. While with modern clear glass this is so subtle as to be unnoticeable, you can see this effect more strongly on older leaded glass windows, which act like a kind of prism, separating the different wavelengths of light.

We can achieve the same effect by simply offsetting the IOR values slightly for each color channel of red, green and blue. This means calculating a different refraction direction for each color and adding them back together to create our image like so:

There we have it folks!

In it's basic form, this is how we achieved the look of our site's hero. Through the magic of raymarching and a little technical shading work we were able to create a performant and interactive experience for the web. We customized the glass IOR, fresnel, and chromatic aberration values to make something more fantastic than realistic, but we hope these comparisons to real-world phenomena have given you an idea about what went on behind the scenes here. 

For more detail on these concepts (and you thought this was technical!) check out these links, which helped us in developing this article:

Distance Functions by Inigo Quilez

Signed Distance Fields by Jason Cole



Go behind the scenes of our interactive homepage hero

Raymarch, Don't Walk

Face Fusion allows for simple photos to infiltrate video for some powerful deep fakes. And although fulfilling my dream to star in Gladiator is just for fun, I have no doubt this tech can deliver valuable work. It's not perfect, but note I did zero editing to this, and I only used one photo. With a few adjustments and additional time, this could be very convincing. 

These tools are bound to increase what's possible in video production, so let's use them for good. Meanwhile sit back and enjoy my starting role with a special guest spot from Sehzat Oner, Director of 3D animation at Y'all.

Here is a little behind the scenes shot for your enjoyment.  The software is running off my video machine in a virtualized server through a web interface.  


Gladiators for good

Face Fusion allows for simple photos to infiltrate video for some powerful deep fakes. And although fulfilling my dream to star in Gladiator is just for fun, I have no doubt this tech can deliver valuable work. It's not perfect, but note I did zero editing to this, and I only used one photo. With a few adjustments and additional time, this could be very convincing. 

These tools are bound to increase what's possible in video production, so let's use them for good. Meanwhile sit back and enjoy my starting role with a special guest spot from Sehzat Oner, Director of 3D animation at Y'all.

Here is a little behind the scenes shot for your enjoyment.  The software is running off my video machine in a virtualized server through a web interface.  




Fun with Face Fusion: David Brungard and Sehzat Oner star in Gladiator!

Face Fusion Fun

DALL-E 3 has some really nice improvements over Midjourney and Stable Diffusion. One of the things I'm the most amazed by is how it's handling human anatomy. For some reason none of these models have been able to get this right, until now.

My request was to create a specific organ made out of Tiffany's glass. Here are just some of the results. 

These are not perfectly correct, but for a comp or the start of a conversation they are amazing. No other AI model can touch this output when it comes to accuracy, even if I like some of the artistic choices by Midjourney a little more.

PSA for all my medical marketing friends, DO NOT USE THESE AS FINAL ART. You should still hire medical illustrators and 3D artists for this kind of work, especially if you don't know what is correct... but for a comping tool this can really help your conversations and hopefully get you better results in the end.


The anatomy of invention

DALL-E 3 has some really nice improvements over Midjourney and Stable Diffusion. One of the things I'm the most amazed by is how it's handling human anatomy. For some reason none of these models have been able to get this right, until now.

My request was to create a specific organ made out of Tiffany's glass. Here are just some of the results. 

These are not perfectly correct, but for a comp or the start of a conversation they are amazing. No other AI model can touch this output when it comes to accuracy, even if I like some of the artistic choices by Midjourney a little more.

PSA for all my medical marketing friends, DO NOT USE THESE AS FINAL ART. You should still hire medical illustrators and 3D artists for this kind of work, especially if you don't know what is correct... but for a comping tool this can really help your conversations and hopefully get you better results in the end.


I asked Dall-E 3 to create specific human organs in the style of Tiffany's glass. The results are gorgeous ... and finally on point. 

Working with Dall-E 3: A real glass act

(accessibility-focused developers, for those of y'all unfamiliar with coding lingo), accessibility is important to us in all of our work. But when we create a site called 

, with 365 ways to do good in this world, it's doubly important that everyone be able to comfortably read and act on the content. Our designers design with this type of web accessibility in mind, but 

what if you wanted to have a dynamic set of colors, without having to manually calibrate each one

The Web Content Accessibility Guidelines (WCAG) sets standards and establishes best practices to ensure web content is accessible to people with disabilities, including those of us simply getting on in years (health is temporary!) To ensure text is comfortably legible for all, these guidelines specify the minimum amount of contrast that text must have against its background. Contrast makes text easier to read. And smaller text must be even more "contrasty" than larger text. We wanted to meet their highest standard, AAA, while also having a new, random-ish set of background/foreground colors each day.

This was a challenge because, while there are many tools out there to tell you the 

 between two colors, there are very few that go the other way. In other words: "given text of some color, give me a background color that would be sufficiently contrasty." Necessity being the mother of invention, we repurposed a few off-the-shelf tools to do just this.

 that would make working with colors easier. Second, to allow us to art-direct the style, we wrote some functions to generate random colors within a given range. Finally, we put them all together with another function to analyze the amount of contrast in each pairing, and shuffle until we get a match. This bit of retrying is necessary as contrast is not a simple matter of dark vs light; colors influence the way we perceive contrast, so for example a yellow might be more difficult to read than a blue of the same brightness, against the same background color.

We may evolve the above into its own module that others may use, but in the meantime this is an approach that has worked well for us, and we hope others will find it useful. Though to the perfectly-sighted person the difference between an accessible palette and a close-but-inaccessible one may seem minor, to those who need it, it's the difference between your site being comfortable vs being a headache, or even unusable at all. Just another way we try to make a daily difference :wink:

As good a11ys (accessibility-focused developers, for those of y'all unfamiliar with coding lingo), accessibility is important to us in all of our work. But when we create a site called Make A Daily Difference, with 365 ways to do good in this world, it's doubly important that everyone be able to comfortably read and act on the content. Our designers design with this type of web accessibility in mind, but what if you wanted to have a dynamic set of colors, without having to manually calibrate each one?

The Web Content Accessibility Guidelines (WCAG) sets standards and establishes best practices to ensure web content is accessible to people with disabilities, including those of us simply getting on in years (health is temporary!) To ensure text is comfortably legible for all, these guidelines specify the minimum amount of contrast that text must have against its background. Contrast makes text easier to read. And smaller text must be even more "contrasty" than larger text. We wanted to meet their highest standard, AAA, while also having a new, random-ish set of background/foreground colors each day.

This was a challenge because, while there are many tools out there to tell you the contrast ratio between two colors, there are very few that go the other way. In other words: "given text of some color, give me a background color that would be sufficiently contrasty." Necessity being the mother of invention, we repurposed a few off-the-shelf tools to do just this.

First, we found a nice little library that would make working with colors easier. Second, to allow us to art-direct the style, we wrote some functions to generate random colors within a given range. Finally, we put them all together with another function to analyze the amount of contrast in each pairing, and shuffle until we get a match. This bit of retrying is necessary as contrast is not a simple matter of dark vs light; colors influence the way we perceive contrast, so for example a yellow might be more difficult to read than a blue of the same brightness, against the same background color.

We may evolve the above into its own module that others may use, but in the meantime this is an approach that has worked well for us, and we hope others will find it useful. Though to the perfectly-sighted person the difference between an accessible palette and a close-but-inaccessible one may seem minor, to those who need it, it's the difference between your site being comfortable vs being a headache, or even unusable at all. Just another way we try to make a daily difference :wink:

Randomly Accessible

Personally, we're not keen on others invading our space. But here our own logos are... invading... so we... blast 'em? (We're not really sure what metaphors are either.) Here's one thing we 

 sure of: Retro video games are fun. And exploding things is funner. Yes, funner.

Originally conceived as a novel UX element for our 

, we ultimately decided the game would be just a bit too odd for site navigation. So we present it here as a stand-alone game, with everything you love about the classic arcade game right in your browser. 

Use your left and right arrow keys to move your ship, and the space bar to shoot.

We're always looking for new ways to bring fun into the web and gamify the everyday.

Personally, we're not keen on others invading our space. But here our own logos are... invading... so we... blast 'em? (We're not really sure what metaphors are either.) Here's one thing we are sure of: Retro video games are fun. And exploding things is funner. Yes, funner.

Originally conceived as a novel UX element for our agency website, we ultimately decided the game would be just a bit too odd for site navigation. So we present it here as a stand-alone game, with everything you love about the classic arcade game right in your browser. Use your left and right arrow keys to move your ship, and the space bar to shoot.

We're always looking for new ways to bring fun into the web and gamify the everyday.

y'all should protect us from Space Invaders. 

Space Invaders

This is a quick test I did with Gen-2 by Runway. The journey started with building the original image in Midjourney. Then I ran the tool a few times to get a 12 second clip (you can now go to 16 seconds). These are exciting times, and don't worry, it's not replacing the artists and animators we need to do final production, but it's another tool that allows everyone to brainstorm and come up with cool ideas. You can see a little hallucinate near the end, but for a few minutes work, this is pretty amazing output!

Ideas take off from Gen-2 Runway 

This is a quick test I did with Gen-2 by Runway. The journey started with building the original image in Midjourney. Then I ran the tool a few times to get a 12 second clip (you can now go to 16 seconds). These are exciting times, and don't worry, it's not replacing the artists and animators we need to do final production, but it's another tool that allows everyone to brainstorm and come up with cool ideas. You can see a little hallucinate near the end, but for a few minutes work, this is pretty amazing output!

This 16-second video was generated by Gen-2 by Runway from an image built in Midjourney. Welcome to the jungle.

Project Runway

We had so much fun testing Wonder Studio while it was in Beta, and it didn't take long to realize that this is really useful storytelling software. If you ever want to create a narrative with a 3D character but don't have an army of 3D and motion graphics artists, this could be the right tool for the job. While I don't think we can say it's ready for final production work just yet, Wonder Studio opens up a world of possibilities for young and amateur creators.

Wonder Studios is now out of Beta for anyone to try at 

AI is an amazing and terrifying tool, and we're jumping right in to learn as much as possible and be better prepared for tomorrow. Never stop learning.

What a wonderful world this could be

We had so much fun testing Wonder Studio while it was in Beta, and it didn't take long to realize that this is really useful storytelling software. If you ever want to create a narrative with a 3D character but don't have an army of 3D and motion graphics artists, this could be the right tool for the job. While I don't think we can say it's ready for final production work just yet, Wonder Studio opens up a world of possibilities for young and amateur creators.

Wonder Studios is now out of Beta for anyone to try at https://lnkd.in/ePj4dg8g

AI is an amazing and terrifying tool, and we're jumping right in to learn as much as possible and be better prepared for tomorrow. Never stop learning.



If you want to create a story with a 3D character but don't have an army of 3D and motion graphics artists, this could be the right tool for the job.

Wonder Studio

It's easy for creatives to feel resistance when AI comes up, but here at y'all, we're excited about all the new opportunities AI offers. So when this very website needed a hero image (or ten), we took Creative Director Mike Foster's beautiful illustrations of our mascot, Yallie, and ran them through a few cycles of the 

 AI tumble dryer to produce the fun goodness below.

First, you give it an image to start from:

Then, you give it a "prompt," which tells it how you'd like to transform the original image. 

We wanted to turn Yallie into a scientist in a lab performing an experiment, while keeping the style fun, flat, and modern. These images were all generated using this prompt: 

colourful flat scientist creature with googly eyes performing an experiment, 2d, playful, airbnb, gradient, aesthetic

Part of the fun is seeing what additional details the AI may add. For instance, we're not sure what's going on with the colorful dots and strange outlines that emerged from Yellow Yallie (but we're lovin' it):

Pink Yallie's peace sign becomes their... drunk buddy?

As silly as these are, we're excited about what they represent for the future of digital art, and the opportunities this truly new medium presents.

It's easy for creatives to feel resistance when AI comes up, but here at y'all, we're excited about all the new opportunities AI offers. So when this very website needed a hero image (or ten), we took Creative Director Mike Foster's beautiful illustrations of our mascot, Yallie, and ran them through a few cycles of the img2img AI tumble dryer to produce the fun goodness below.

How does this work? 

First, you give it an image to start from:

Then, you give it a "prompt," which tells it how you'd like to transform the original image. 

We wanted to turn Yallie into a scientist in a lab performing an experiment, while keeping the style fun, flat, and modern. These images were all generated using this prompt: colourful flat scientist creature with googly eyes performing an experiment, 2d, playful, airbnb, gradient, aesthetic

Part of the fun is seeing what additional details the AI may add. For instance, we're not sure what's going on with the colorful dots and strange outlines that emerged from Yellow Yallie (but we're lovin' it):

Pink Yallie's peace sign becomes their... drunk buddy?

As silly as these are, we're excited about what they represent for the future of digital art, and the opportunities this truly new medium presents.

AI Yallie

We continue to be horrified by the Russian invasion of Ukraine. In our own small way, back in March of 2022, we wanted to express our solidarity with the people of Ukraine, and to help others do so as well. Around the same time, 

 was taking over our newsfeeds with colorful results posts, so why not try to harness a bit of that and turn the 🟨 and 🟩 into 🟨 and 🟦❔

 is much more important: to show your support for Ukraine. To try it out, first play today's 

. And while you're there, check out this list of 

 and take action today. ​💙🇺🇦💛​

We continue to be horrified by the Russian invasion of Ukraine. In our own small way, back in March of 2022, we wanted to express our solidarity with the people of Ukraine, and to help others do so as well. Around the same time, Wordle was taking over our newsfeeds with colorful results posts, so why not try to harness a bit of that and turn the 🟨 and 🟩 into 🟨 and 🟦❔

How?

For this one, the why is much more important: to show your support for Ukraine. To try it out, first play today's Wordle then paste your results into https://suppordle.com. And while you're there, check out this list of 30 Meaningful Ways You Can Help Ukraine and take action today. ​💙🇺🇦💛​

Suppordle

At Y'all, we're always exploring innovative ways to create engaging and interactive experiences for our clients. Recently, we've been captivated by Google's MediaPipe framework and its potential to revolutionize our conference installations. MediaPipe's advanced machine learning capabilities enable real-time, cross-platform solutions that enhance attendee engagement and interaction. From interactive displays that recognize gestures to augmented reality features that bring presentations to life, we're excited to leverage MediaPipe to create unforgettable conference experiences.

Here are three of the features we're most excited about:

One of the suite of offerings that Media Pipe includes is the ability to recognize hand gestures. By incorporating hand gesture controls, we can create intuitive experiences that go beyond traditional interfaces. Imagine navigating through a virtual exhibition with a simple wave, selecting options with a flick of a finger, or manipulating 3D objects in real-time with natural hand movements. With MediaPipe's precise and reliable hand tracking, we deliver responsive and dynamic installations that provide a truly memorable experience.

In the demo above, we're exploring the use of custom swipe and pinch gestures, as well as the inbuilt hand gestures that come with the open source framework. These include thumbs up and down for measuring sentiment, closed fist and open palm for recognizing a grabbing gesture, and just for fun, a "victory" (FKA peace?) sign and love sign. Any of these gestures can be leveraged as a signal to select or navigate, instead of relying on the clunky old mouse click, touching a physical screen, or expensive custom hardware.

Pose estimation technology is another feature of Media Pipe. This feature allows us to track and analyze human body movements in real-time, enabling engaging applications. Attendees can interact with digital displays through their body movements, participate in immersive fitness demonstrations, or control virtual environments using natural body gestures. By leveraging pose estimation, we create experiences that are highly intuitive and engaging, transforming passive viewers into active participants.

This technology enables our displays to recognize and respond to various objects in real-time. Imagine an attendee holding up a product and instantly receiving detailed information on a nearby screen, or using everyday items as interactive tools to control digital environments. With MediaPipe's accurate and efficient object recognition, we can create dynamic installations that respond to the physical world, providing an innovative and captivating way for attendees to connect with our content.

This is only scratching the surface of the suite of Machine Learning tools that dovetail nicely with conference and instillation work. Armed with any camera and a strong computer, we can create immersive experiences that dazzle users and get away from the old ways that we'd interact in digital worlds.

At Y'all, we're always exploring innovative ways to create engaging and interactive experiences for our clients. Recently, we've been captivated by Google's MediaPipe framework and its potential to revolutionize our conference installations. MediaPipe's advanced machine learning capabilities enable real-time, cross-platform solutions that enhance attendee engagement and interaction. From interactive displays that recognize gestures to augmented reality features that bring presentations to life, we're excited to leverage MediaPipe to create unforgettable conference experiences.

And all you need is a camera!

Here are three of the features we're most excited about:

Hand Gestures

One of the suite of offerings that Media Pipe includes is the ability to recognize hand gestures. By incorporating hand gesture controls, we can create intuitive experiences that go beyond traditional interfaces. Imagine navigating through a virtual exhibition with a simple wave, selecting options with a flick of a finger, or manipulating 3D objects in real-time with natural hand movements. With MediaPipe's precise and reliable hand tracking, we deliver responsive and dynamic installations that provide a truly memorable experience.

In the demo above, we're exploring the use of custom swipe and pinch gestures, as well as the inbuilt hand gestures that come with the open source framework. These include thumbs up and down for measuring sentiment, closed fist and open palm for recognizing a grabbing gesture, and just for fun, a "victory" (FKA peace?) sign and love sign. Any of these gestures can be leveraged as a signal to select or navigate, instead of relying on the clunky old mouse click, touching a physical screen, or expensive custom hardware.

Pose Estimation

Pose estimation technology is another feature of Media Pipe. This feature allows us to track and analyze human body movements in real-time, enabling engaging applications. Attendees can interact with digital displays through their body movements, participate in immersive fitness demonstrations, or control virtual environments using natural body gestures. By leveraging pose estimation, we create experiences that are highly intuitive and engaging, transforming passive viewers into active participants.

Object Recognition

This technology enables our displays to recognize and respond to various objects in real-time. Imagine an attendee holding up a product and instantly receiving detailed information on a nearby screen, or using everyday items as interactive tools to control digital environments. With MediaPipe's accurate and efficient object recognition, we can create dynamic installations that respond to the physical world, providing an innovative and captivating way for attendees to connect with our content.

Closing Up

This is only scratching the surface of the suite of Machine Learning tools that dovetail nicely with conference and instillation work. Armed with any camera and a strong computer, we can create immersive experiences that dazzle users and get away from the old ways that we'd interact in digital worlds.

Exploring Google's Media Pipe for creative installations

Welcome to the lab

Lessons of Yallie

Blood Vessel Voyage

Yallie Likes to Move it Move it

Yallzilla

Alchemicals

Notes of Forest Floor

V for Vertex

🔥YＯLＯ🔥 in a Different Way

Raymarch, Don't Walk

Face Fusion Fun

Working with Dall-E 3: A real glass act

Randomly Accessible

Space Invaders

Project Runway

Wonder Studio

AI Yallie

Suppordle

Media Pipe Dreams

Welcome to .css-3nsmuw{--tw-text-opacity:1;color:rgba(var(--brand-cyan), var(--tw-text-opacity));}the lab

Lessons of Yallie

Blood Vessel Voyage

Yallie Likes to Move it Move it

Yallzilla

Alchemicals

Notes of Forest Floor

V for Vertex

🔥YＯLＯ🔥 in a Different Way

Raymarch, Don't Walk

Face Fusion Fun

Working with Dall-E 3: A real glass act

Randomly Accessible

Space Invaders

Project Runway

Wonder Studio

AI Yallie

Suppordle

Media Pipe Dreams

Welcome to the lab