HandyCon 2024 | Day 2 - Spatial Domains - YouTube
https://www.youtube.com/watch?v=ee_M33RXZfQ

Transcript:
(00:03) [Music] all right looks like we're live hello handy con 2024 this is uh Christopher Moore you might remember me from the flamingo handshake auctions back in the day and I'm back now with Neils pill uh the founder and CEO of OI labs and the post mesh foundation and um I linked up with Mike to bring him to this conference today because uh all of us are domainers and he's working on a project that deals with um augmented reality and spatial Computing and the topic today is going to be spatial domain uh as the interfaces of the future
(01:03) change it might make uh domains less valuable so it's kind of a Counterpoint to probably a lot of stuff you've heard on the uh previous sessions and other domaining things so I'm gonna kick it off to Neil's and if there's time towards the end we might have some U time for questions but if not I'll let Neil um go on about spatial Computing uh finding location and navigating online or in space in the future Neils I'll hand it off to you all right hey good morning from Hong Kong um yeah I'm here
(01:44) to talk a little bit about how spatial Computing is likely to change some of how we expect the internet to work normally uh and by spatial Computing I don't mean fancy headsets although headsets are a part of the spatial Computing movement the real thing that is happening with spatial Computing is that uh computers are starting to understand physical space you could say that spatial Computing really is the art of teaching computers to understand physical space when apple called The Vision Pro uh a spatial computer in in
(02:21) the intro announcement Tim Cook actually said uh just as the Macintosh introduced us to personal computers and the iPhone introduced us to mobile computers The Vision Pro is an introduction to spatial computers it's not that it's a headset that makes it a spatial computer the thing that makes it a headset is that it understands physical space and when a computer understands physical space you can start putting digital information in physical space and this has huge implications of course for things like
(02:51) augmented reality and the metaverse built on augmented reality but also for uh drones robotics self-driving there turns out that computers don't have a good understanding of their place in the world normally so spatial Computing has been one of the um secret buzzwords uh before uh even Apple released the headset if you look at the uh seven big tech companies in the in the top 10 companies by market cap uh all of these companies have invested very very heavily on the order of billions in the last few years into spatial computing
(03:31) if you think of uh Tesla as an AI company for example uh one of the things that makes Tesla a very competitive AI company is the quality of their spatial Computing one of the biggest assets they have is that they have millions of cameras out in the world reconstructing an external digital memory of what the world looks like so that their cars their robots can navigate the world and when information is stored in physical space when it's indexed by physical space information Discovery uh ends up being different what we are
(04:06) building at the post mesh is something that we also call domains uh a domain is a representation of a physical space self-hosted the way a website can be self-hosted if you want it to be uh and when you connect to a domain the computer gets an immediate understanding of its physical surroundings it's like you can connect to a sense of space but not only that within the domain uh you can also share spatial data like observations what am I looking at what am I perceiving so that any individual computer can understand the scene better
(04:44) than they could from just their uh own sensors so uh we believe that decentralized domains Sovereign domains that you own and you can invite devices to come visit you and uh a protocol on top of that that allows robots machines computers to move between domains uh to hop on and hop off into these collaborative spatial Computing clusters uh we think that's a necessary part of the future of the internet the alternative uh would be uh pretty nightmarish actually since spatial Computing is necessary for this future
(05:21) spatial internet and a positioning Services necessary if we are to imagine a centralized positioning service a company like Google or Microsoft helping every computer understand where they are that would be quite akin to like one company uh owning the internet I uh am a very strong believer that that spatial Computing and everything that comes with it um augmented reality in particular for for me personally um is going to be as impactful as the previous big computer revolutions we've seen we can expect spatial Computing to have as big an
(05:57) impact as the rise of mobile computers we can expect spatial Computing to have as big an impact as the rice of personal computers we can have spatial Computing to have as big an impact as the internet just like computers touched every industry eventually and mobile computers touched every industry eventually spatial Computing will touch every industry eventually the trend we are in history now that has just begun is that the world's information is getting indexed by physical space so that spatial computers can navigate the
(06:28) spatial internet and uh our concept of domains as a physical representation or a digital representation of a physical space that you can collaboratively perceive in a trustless way backed by a blockchain reputation system we think that's going to be an essential part of the internet itself in in the coming years cool um I can give a demo of um a spatial an example of a spatial domain that we just uh tested a day or two ago let me see if I can get it on the computer here um if I can get a video so while Chris is loading that
(07:12) I'll explain again that a domain is a self hostable digital representation of a of a physical space that has been made accessible and machine readable uh in an interoperable way so that many different kinds of devices many different kinds of applications can use this digital representation in real time collaboratively to recent about space so uh our uh our company is focusing on a couple of applications on top of the post mesh on top of domains for for example helping retailers um and Retail staff navigate
(07:50) the store better when you have a digital copy of your store uh that is navigable by AR even a first day employee can find every product in the first 20 minutes of them working there this is Meaningful of course if you've ever worked in in in retail as I have you know that the staff turnover in retail is incredibly High a fun back of the napkin calculation you can do is uh that Walmart uh spends around a hundred million do every year just on first day salaries just first day salaries $100 million so being able to put digital
(08:28) information in physical space u is bound to be really really disruptive for Industries like retail like warehousing like Hospitality really any kind of physical business um and a big driver of this is artificial intelligence because AI is very soon going to be both the best and the cheapest worker for a lot of tasks but AI can only understand digital things and a physical business is not a digital thing so without something like a post mesh domain that allows the physical space to be collaboratively understood uh in real
(09:04) time uh the AI does not even have access to join that kind of Workforce okay the the video I have is on my desktop and my uh uh browser doesn't allow me to share so sorry about that I'll try and get it into the group later after the talk but um uh maybe I'm allowed to share my screen and I can can show something here yeah I was thinking of the blimp video sure I can pull up the blimp video we're right here in our Discord now looking under the hood seeing all of our Dirty Secrets looking for the creative
(09:48) channel so here for example is this is what Chris wanted to show a city scale experience um where augmented reality information is accessible from for visiting devices so this is not a scene that is invented on my phone uh this is part of a domain we're here in cun Bay so by being able to calibrate into this scene the post mesh helping me get my position in this scene uh I can see this digital scene um so uh in a sense uh postmes domain uh can very meaningfully be thought of as virtual re real estate in the same way that traditional domain
(10:31) names are virtual real estate and currently the way that you access that is there's that QR code on the lamp post and that uh creates that's what we would call The Domain that tells the device that you're in that space and then the blimp and the flying objects spaceships whatever those were are um things that exist in that domain so um the the big why are there QR codes right the the big problem of of positioning is that unless we live in a simulation there is no absolute coordinate system that you can tap into
(11:14) um things like the GPS are way too imprecise like not even close to precise enough to doing an experience like this you want to have accuracy of a few centimeters in positioning and GPS in a big modern city like Hong Kong WR is lucky if he can put you on the same city block the the reason for this is GPS is actually a line of sight based technology you need to be able to see a couple of satellites in the sky for it to work well and very rare to find a piece of of land in Hong Kong where you can see a couple of satellites uh from
(11:47) where you're standing so uh the GPS is already dead people just haven't really woken up to that fact um so new Positioning Systems are needed this is is why the the big seven tech companies have invested so much in spatial Computing and a lot of them have invested in centralized visual Positioning Systems like uh Google has a very very amazing one for example called the spatial API but the way centralized visual Positioning Systems work uh is that you share your camera feed with the centralized provider you tell them what
(12:23) you're looking at and they compare that to the external memory of what the scene looks like that's stored in their class and they compute for you uh where you are so the trait is if you tell Google what you're looking at Google will tell you where you are the post mesh is an attempt to uh invert this workflow so uh you can instead calculate your own position and ask data from the external memory so that you don't have to share what you're looking at Instead The Domain has to share information with you
(12:55) so that you can do your own privacy preserving computation of course if you're an ad driven company that's not what you would want to do but that's also why the decentralization movement is so meaningful and why deepin is so meaningful uh we believe that a deepin a decentralized machine perception Network which is essentially what the post mesh is uh is going to drive the physical discovery of information in the spatial internet era and uh with with that said so the the post mesh is live today we have over
(13:29) 300 nodes operating um we recently open- sourced the um node uh code there'll be more o open- Source uh releases in the future uh we're going to be releasing I think in the next month the domain service and the so you can create domains in your house if you have an nft you could like put it uh create create an augmented reality nft Gallery in your office your shop your home wherever um these are some of the things we have coming up we also have a token launch um at some point in the next few months uh we raised a bunch of money in
(14:10) uh 2021 we have operating um Tech U didn't want to launch the token until that was out um additionally we I know there's a lot of developers and Technical people in the hns and handshake community that also um value decentralization and also love to create things so we have a thing called the conjure kit SDK where um you can create your own augmented experiences and run them on the post mesh and use these domains to basically um start to interact and build in the spatial internet of the future I'll let Neils
(14:51) jump in if there was anything there that you might want to add uh minor note that we're retiring the conri kit branding uh to keep simple uh it's a postmes SDK so you can get the postmes SDK and build your own applications that make use of domains that make use of the uh International postmes Network that we've already built up uh you'd be joining uh other companies uh small and big like animoca building the future of spatial experiences and again what's so special about the solution is that it allows the
(15:25) spatial Computing to be collaborative so you can see things together you've been able to see things in AR for many many years but what you haven't been able to do until now is effortlessly show someone else on their device in the exact same place where you are seeing the content so the postmes allows for the first time for you to put AR art on your wall and other people will see it other people will see it and we uh thought back in 2021 that this was going to be really impactful for nft as well in the sense that nfts was the
(16:04) first part of helping the Public's imagination understand that digital things can be real right digital things can be real uh we think that when you see digital things together in physical space uh the way shared augmented reality does that really drives the message home it really helps you feel like this digital thing is real um so yeah the postmes now it's real uh we have a question from the audience Neils um yes screw Google but apart from GPS aren't devices already more relying on Wi-Fi and ble scanning and comparing
(16:41) that to open IP location databases absolutely devices are doing that but even with uh Wi-Fi scanning you are not getting positions better than one or two meters uh typically which is nowhere near good enough for something like uh like robotics or shared augmented reality you can do Wi-Fi triang a uh in fact uh you you probably already know that both your iPhone and your Android is constantly listening to what uh Wi-Fi networks you are in range of and using some simple heuristics based on uh Network strength to estimate how far
(17:16) away you are from that Wi-Fi node and that will do better positioning for you but but again at scale in a city like Hong Kong it works worse um and it's still even in the best conditions not precise enough so Wi-Fi is very helpful uh it's better than GPS but it's also not good enough you know the email was better than the fax machine but a chat is better than an email and we're we're going through that Journey again also Neils it's quite interesting the uh overlap of AI blockchain and
(17:56) spatial Computing um maybe you can go a little bit deeper on how Ai and special Computing um can interact today and in the future and what that means so when uh Yan laon the head of AI at um meta went on Lex Friedman earlier this week I think he said that AI is still missing four important things for it to truly become as useful as he would like one is a a sense of physical space uh the the second uh is a memory um so it can learn from its experiences the third is planning and the fourth is reasoning starting up here uh with the
(18:40) the sense of physical space uh this is what I've already hinted on uh machines can only understand the digital information uh without spatial computer without spatial Computing an AI just does not um understand the fiscal world if you want a robot that can look at the world and understand the world you're in the field of spatial Computing already collaborative spatial Computing is the idea that two machines should be able to form an opinion of the world together which is one of the the the big big technical challenges it's relatively
(19:17) simple to make a robot that can on its own kind of build up a a map of its surroundings and stop bumping into the same wall over and over it's a lot harder to make that map be collaborative with other robots of the same kind and it hasn't been done yet to make a collaborative map that can be accessed by any robot and that's what the post mesh is working on um excellent do we have any uh questions from the the audience I I see someone asking if if this can work with um handshake uh one of my uh motivations
(19:59) for bringing Neils on here today is I know a lot of you are technically inclined and doing stuff uh in the decentralized web using hns domains and things like ipfs and uh decentralized uh storage and uh I was hoping to bring Neils on here just to introduce this topic and uh maybe somebody has ideas for building out on H hns and completely uh decentralizing with the spatial Computing and the apis that we offer so um Can this work on handshake uh that's for the community community to decide and I'm sure if uh
(20:39) anybody had some ideas and was building on that we will have Grant programs coming up in the future so uh reach out to me or reach out to Neils uh we're looking forward to support these type of projects um so that is an opportunity that is definitely open um I'm G it's very important unironically that the positioning framework of the future is decentralized and not owned uh by a corporation that is unironically of civilizational importance so it's important for us to come speak to the decentralization community and
(21:14) communities like this community uh to spitball for the sake of you know really the thriving of our civilization how can we make sure that the spatial internet which is for sure the next version of the internet how do we make sure that this version of the internet does not end up being owned by Google so far we've really been uh just barely getting by uh in keeping the internet free uh companies like meta and Google and and uh Twitter have way too much power over us already and I really hope that the decentralization
(21:54) community uh now in the age of AI and the age of spatial Computing uh will start rallying around the importance of the decentralization of this Tech stack because I genuinely believe it's more important to decentralize spatial Computing than it is to decentralize the bank um people can do more harm to us as a civilization by always knowing what we're looking at than controlling the money supply maybe go into more details on how the uh the big Tech looking through our eyes when these uh devices become glass
(22:29) and maybe also talk about the potential that we may have the opportunity to do with the post mesh to create augmented reality experiences via this technology without cameras or um any of that uh invasive again for really fine grained positioning for really fine grained positioning every major tech company without exception every major tech company has adopted the hypothesis that visual positioning is the way to go and visual positioning means that you will be comparing your camera feed or information derived from your camera
(23:04) feed to an external memory of what the world looks like so again using Google as an example they built a 3D model of the world using the billions of photos taken by Google street view the the cars uh and that's super super impressive like it's it's one of the most in my opinion one of the most impressive engineering projects that Humanity has ever done but it's also really really scary because what it is is you tell Google what you're looking at they will tell you where you are and this is how
(23:35) companies like Google and Niantic and Microsoft imagine the future of AR glasses working that in a world where you wear AR glasses all day they know where you're looking all day and not only do they know where you're looking but because uh they need to look at your eyes for other reasons related to better rendering quality and uh battery consumption and helping you with the interface face they will even know how you feel about what you're looking at and I think it's important to remember that when meta was asked if they would
(24:08) ever uh use this data sell this data to other people they preferred not to comment like this is this is a real threat this is happening now this is already being built this surveillance technology has already been deployed and it's already working they're just driving to get adoption now so as the decentralization community we have to not only build a better product which I think we are but we also have to drive adoption faster than the big tech companies uh and that's only something we can do together as a
(24:39) community okay we have uh three minutes left and there's uh two questions uh from the audience one is uh spatial and I'll I'll drop them both and then this will be basically wrapped up after this uh spatial Computing doesn't sound like it cares much about borders do we think that large tech companies will run into jurisdiction problem since they are Incorporated within geographical territories forcing them to create or adopt dweb Tech and then the second one is why are you not mentioning Apple any specific reasons so
(25:13) um I don't think that the issues that corporations will face with legislation will be enough to force them to really decentralize they haven't had to decentralize that much yet like yes Google will have many data centers in in many different jurisdictions but Google having many data centers to me that's not decentralization uh I don't think there are sufficient legal challenges to the corporations to to really make them adopt dweb um I didn't mention Apple very much because Apple doesn't do collaborative
(25:48) spatial Computing at all apple is the good guy so far in the spatial Computing race in that they don't give anyone even themselves access to the camera feed um so we'll see where Apple goes from here uh Apple has made a visual positioning system that can work for your phones like you can use it uh but the Vision Pro doesn't use it uh and I think that's very meaningful I'm hopeful that Apple will continue to drive uh responsible standards for privacy but also the way that Apple has solved this right now
(26:20) with the Vision Pro is essentially making the Vision Pro incapable of collaborative spatial Computing at least for now that's why you have never seen a demo of two people wearing Vision Pros seeing the same thing at in the same place uh that is uh so far not possible to do on the Vision Pro perfect and our time is up uh thank you for joining the conference Neils thanks Mike for creating the opportunity for us to come on here and talk about spatial domains and uh the future ahead and I hope some of you in the audience
(26:54) uh or those that watch after the fact uh reach out and get involved build on the post mesh you can support us with nodes build AR experiences we're open reach out on Twitter or on Discord uh we are discord.gg alivers a UK k i v RS e I'm brood sugar like blood sugar but with an r on Twitter my DMs are open uh we are aabs docomomo all right p o m.
(27:29) org over and out you guys can close the [Music] chat