VR Experiments

April 27th, 2016 No comments

While giving demos for MaximumVR I had a few people mention to really feel like a Godzilla monster they wanted to stomp around the city.
So to add feet Kate used some old Vive Dk1 controllers and attached them to rubber boots:

Had to use old Vive Dk1 controllers rather than the pre/consumer versions since those have a bandwidth limit of 2 (they go through the headset, while old ones have a separate connection).
We originally just tried strapping them on the feet directly, but this had the problem of not knowing where the ground was exactly (your feet would be at a different position depending on where you strapped them to your feet, plus the angle you strapped them on would need to be perfect). Giant rubber boots ended up working well also since players just naturally felt ridiculous wearing them.

No way players will actually be able to try this yet – but hopefully full body tracking will eventually be a thing!

One of the main Vive experiments I wanted to try since getting the first dev kit was interacting with fluid and soft bodies. I wasn’t sure of the general game idea, but I knew that just interacting with non rigid bodies would be incredibly satisfying.

To do this I had to grab the Flex branch of Unreal and compile it, then just added standard motion controllers. My plan was to make a pancake simulator game (just since the most satisfying liquid was one with a lot of viscosity, and pouring it then moving it around in a pan was fun). I knew the Flex liquid simulation was pretty limiting game wise (no collision event data, no way to serialize, and can only change properties for the entire group), but just messing with the liquid in anyway would be enough of a game for me.

But, I got tired of dealing with the Unreal engine. I am glad I took a weekend to learn Blueprints and get a C++ refresher, but the time it would take me to go farther with this project wouldnt be worth it.

Categories: Modbox, VR

Our VR game: Modbox

December 11th, 2015 No comments

I am pretty excited about this project.

I did a interview with Killscreen here.

Categories: VR

Why there is something rather than nothing

December 1st, 2015 1 comment

WaitBuyWhy has a great article about the question of ‘why there is something instead of nothing’, I particularly liked this part:

Religious people have a quick answer to “Why is there something instead of nothing?” I’m not religious, but when I’ve thought hard enough about it, I’ve realized that it’s as plausible as anything else that life on Earth was created by some other intelligent life, or that we’re part of a simulation, or a bunch of other possibilities that would all entail us having a creator. But in each possible case, the existence of the creator still needs an explanation—why was there an original creator instead of nothing—and to me, any religious explanation inevitably hits the same wall.


So to give my own attempt to explain why there is  ‘something rather than nothing’, I want to give one that that doesn’t require a first cause – that the original creator of reality is reality itself – and it exists for this purpose. A self defining system, a Strange Loop.

I not only mean this as the ‘how’, which is a pointless circular definition, but as the ‘why’.

The main thing that would need to be the case for this to make sense is the universe being entirely digital and computable. This is not a strange idea and has been adopted by a large number of  physicists such as Max Tegmark’s with his book ‘A Mathematical Universe’ (who argues in his Mathematical Universe Hypothesis that reality is not just computable, but a static mathematical structure), or in the case of digital physics the writing of Stephen Wolfram. You can go over the history of Digital Physics to see its evolution over decades.

Essentially the idea of digital physics is that for the universe to be computable it must be able to be described digitally. So while treating spacetime as continuous still has use (like treating water as a continuous fluid rather than tons of individual particles) – it’s convenient as an approximation, but at the most basic level spacetime is actually discrete – like a grid rather than a continuous medium. Where grid positions are separately by the smallest possible length (which might be the Planck length)

If reality is computable, then all existence is actually just information. If you look at an apple, the properties you think it has are not just 1s and 0s. It has taste, color, smell – but these are all higher level properties, and actually at the lowest level it’s entirely elementary particles – electrons, quarts, things that can be described entirely mathematically. The higher level properties are emergent from these base mathematical entities. A quark has no ‘smell’ or color, it simply has a number of mathematical values.

If reality is computable, it can be simulated. If it’s all information, we can take this information and it can be calculated. Digital Physics then says the universe can be replicated perfectly by a computer simulation.

If the exact formulas were known, the theory of everything, we could simulate reality in a perfect simulation.

If the simulation of reality is a perfect simulation, and reality is in fact just mathematical, then it is actually just as real. And not only is it real, it’s actually exactly EQUAL to reality. If reality and the universe are just information, then the simulation of the universe is actually the exact same thing as the universe itself.

If the universe was simulated inside itself, it would be creating it’s own existence. Making A=A. It would be the ‘how’ there is something rather than nothing.

Max Tegmark in his Mathematical Universe Hypothesis argues that reality is part of a mathematical structure that is completely defined, and us experiencing time is really an illusion, because we are just part of this already defined static structure. His explanation for why there is something rather than nothing is really because ‘there is everything!’. Every mathematical structure exists, he argues,

Not only is it depressing believing that everything is predetermined and free will doesn’t exist, or that my sense time is completely an illusion, but it also seems like another infinite regress answer to the question of existence.

So rather than Max Tegmark’s hypothesis where everything that can exist does exist – what exists is what can be defined. And what can be defined is what can define itself. Time is calculation, and this calculation is defining reality.

So if the universe needs to define itself, how would it do it? And how does this explain the ‘why’?

It’s possible us humans are a good candidate to create this simulation – after all it’s pretty clear our technological progress is going more towards simulation technology than anything else.

So we would not only have to simulate the starting of the universe and all of it’s properties in a computer program – but we would need to assure that the possible configuration where the universe simulates itself is found.

This computer program would need to search for the possible universe configuration to find the ‘answer’, the configuration where it would be able to simulate itself.

In computer science this type of search problem is called a ‘NP hard problem’. Where it’s easy to verify an answer is correct, but extremely hard to find a solution for. So hard you literally have to pass through every single option – there is no way to predetermine.

A common simple example of this type of problem is:

Suppose you were building two large towers by stacking rocks of various sizes, and you needed to make sure the towers are the exact same mass. To check the answer to that is simple – you just add up the rocks and test if they are equal. But to find the correct configuration isn’t that simple – you need to go through every single possible arrangement of the rocks in order to find the answer. With 100 rocks, that’s 2 to the power of 100, meaning the amount of configurations is a number with thirty digits, and with each additional rock that number grows exponentially.

So even knowing the ‘answer’ and what our simulation would need, we would still need to go through every possibility in a search.

I think this begins to explain the ‘why’. Why reality is setup the way it is – because it’s doing exactly this, it’s searching through all possible logical configurations.

According to inflation theory different possible physics laws are realized in the multiverse (over the string theory landscape, which could be seen as adding varying ‘rules’ to the initial search)  – and according to quantum mechanics and the many worlds theory all possible logical interaction happens (in parallel worlds, which could be described in computer science terms as a ‘breadth first search’).

When I am setting up a computer program to simulate something and search for emergence, this is exactly how I would do it. There would be randomness in the programs initial conditions, fundamental randomness to ensure there wouldn’t be uniformity throughout, and breadth first branching so all possible interactions are realized.

In the universe time is the computation, string theory variance it’s different initial conditions, quantum mechanics it’s fundamental randomness and ‘many worlds’ it’s branching. So how we would ‘search’ for this configuration in a computer program is just like what reality is actually doing

That’s the reason for why reality has the rules it has – so that it can define itself. Reality is as complex as it needs to be. Why it’s perfectly tuned to provide emergence – and why the rules it has are completely necessary.

I would argue this is a good outlook to have in comparison to Tegmark’s Mathematical Universe Hypothesis. The future is then undefined. If we were able to put the current state of the universe in a simulation and predict the future, that is not predetermining the future, that is creating the future in the same way.

Not only is our free will still a possibility, but also there is the possible outlook that life exists entirely for this purpose. Humans are special – we clearly have never found any evidence of anything like us. So maybe we have the chance of creating this simulation, of defining existence.

My insane proposal is then for humanity is to build this simulation – and recreate our existence – so that everything can exist. Create the self referential strange loop. So basically everything I am saying is the plot of The Last Question by Isaac Asimov.


Some random notes:

– When it comes to the question of how do we physically do this – we’ll it might be impossible and none of this actually makes sense – or maybe there is some hope in quantum computers computing this in a reasonable time. It’s also the case that maybe we don’t need a revolution in computing – it would just take a hell of a lot of time to compute. It doesn’t require speed – the universe could take billions of years to calculate 1 second of a simulation and still be possible. Maybe it would take having a computer drift in space as the big freeze takes place calculating the simulation for near eternity…

– I am writing this with a lot of assumptions: that P!=NP (which is almost definitely the case but not proven), many worlds theory of quantum mechanics is correct, and that inflation theory is correct

– What I call the ‘universe’ a few times here should really be the result of the ‘post-inflationary bubble’, but I’d rather this be clear than accurate.

– Tegmark has a brief part in his book about why the universe isn’t a simulation. Part of it is that time isn’t global in reality (as most simulations would treat time) – but there is no reason that this needs to be the case. All modern engines definitely use time globally, but it’s pretty easy to imagine making a 3d simulation with independent time and relativity in mind. Making a game engine where the physics time step is not global would actually be a interesting project.

– I don’t think it’s a requirement that the universe needs to be discrete to be computable. I could certainly represent continuous physics with a simulation – and only compute exact values when when measured. It is definitely possible to represent a uncountably infinite space in a digital simulation. It would just mean you can keep calculating into further precision, this is not finding its true value but defining a value out of a continuous function. You could have rules that are continuous functions but are ultimately in a discrete computable space.

– The argument that Max Tegmark also had is that there is no need to ‘run’ a simulation, if a universe can be described by a consistent mathematical description then it ‘exists’ and is ‘real’. The digital physics idea is that it needs to be computed to be defined, rather than just described. It seems impossible to resolve what would make something ‘real’ – but one example I would give is that if I emulate your brain on a hard drive, and let it sit there – would that be enough for your mind to exist? The view with digital physics is that it needs to be ‘ran’ and computed. I don’t know what makes things real, but I would argue its this information processing, and that this computation is needed. Our consciousness is what it feels like for our minds to be calculated.

Categories: Random

Vive VR Experiments

November 25th, 2015 No comments

I was lucky enough to be one of the developers to receive a HTC Vive kit from Valve (after some begging to my only Valve contact).

I tried it for the first time at PAX 2015. The previous version I tried at Steam dev days, which I wrote about here, had a completely different setup and no input system. I couldn’t see it being used for games very well with no input, but it definitely made me realize the potential of room scaled VR at the time.
For this new demo I tried 5 games. Some of which were disappointing in how they didn’t use the Vive’s amazing input system to it’s full extent (I imagine since they were started / designed before this input was even possible) – so TiltBrush was the application that really made me see the Vive’s full potential. I felt more in control of the painting in Tiltbrush than I have ever had in any input device – better precision than even a mouse.

As soon as I left the demo I knew -exactly- the game I wanted to make, and spent the rest of my trip thinking about it and planning it. I’ve always enjoyed physics building games so this idea is just a extension of that – but I think with the Vive’s input it can really be taken to a new level. I now see it as a general holodeck builder in a sense – you build a environment/game and then share it with others.

I previously had no interest in doing VR work for a variety of reasons – mostly because of the lack of a good input system. I felt if all I could do was cinematic work, then the best games for it would be ones with large teams of artists, basically high budget film like experiences. Without a good input system I couldn’t imagine a systems based design – so being able to make a good product with a low amount of resources didn’t seem possible.
I also had no interest in having another shitty dev kit experience like I have had with all Oculus hardware. I barely even tried to develop anything with Oculus, mostly just running demos, and it was a horrible setup experience. Having to drag a window to another monitor while looking into the Oculus headset is something I never want to do again. Their old Unity integration meant quickly iterating on game ideas was also incredibly painful – usually having to build then test it. Sudden frame rate drops and crashes (both of which are common when developing) also meant constant nausea.
So not only did the Vive solve my problems with input, it’s integration with Unity is fantastic. I can test things in the editor itself, and even adjust the environment in scene view and hot swap code while the Vive is still running. Also when there was a frame rate drop or the application crashed it wouldn’t bring me back to a desktop (which is painful on the eyes), rather a blank VR room.

Before I started on my dream physics VR Vive game though I wanted to quickly try a project to get an idea of how to use the development kit. So within a few days I made this Godzilla sorta simulator using the game assets of another in development Alientrap game:

It took a bit to get the right scale – at first I was imagining the player as a much bigger monster, towering above everything, but I realized it was a lot more fun to be about the same height as the buildings you were destroying.
Instead of monster hands, or a hammer to destroy the city I decided to add a Morning Star type weapon. This was because there is no way to give tactical feedback to the player – if they have a hammer in their hand and they smash a building with it, it will go right through it with no physical feedback. It was important the enviroment feel tangible for destruction to really mean something. So with a Morning Star, the player would have to wind up to get a large force, and would have the feedback of the buildings stopping the morning star. So the world would still feel like it took effort to destroy.

Hopefully I’ll be able to show off our Vive game soon – kind of just waiting to see the best way to announce it. I should probably just give up on the idea of a big announcement and be entirely open with development – but we’ll see

Categories: VR

(very) short story: Kepler’s Strange Loop

August 19th, 2015 No comments

Wrote this quickly for Terraform’s short sci fi story contest (had to be about AI, 50/100+ years out, less than 2000 words)



As Kepler stared blankly at his computer screen, he had a strange thought.
If this code base I am working on is of myself, and the bug is in my own mind, is it even possible for me to find a solution?

“I understand the problem” Kepler’s supervisor said, clearly not understanding at all. “The error lies in the system’s logic functions. It’s referencing itself, leading to a recursive loop. Make an exception and just don’t allow this” the supervisor said, motioning with his hands as if he had found the obvious solution.

Kepler knew the supervisor wasn’t capable of helping, but it was standard protocol to alert the supervisor in case of a problem. He wanted to just avoid this error – to move on to another project entirely, but without the supervisor’s approval he had to stay with the task he was assigned.

“You can’t just remove self awareness from the system.” Kepler was frustrated in having to explain the purpose of this software once again. “This is the MetaMind programming system – designed to program software. How would it be capable of programming if it wasn’t able to know about itself? How would I be able to have this conversation with you at the moment, without being able to refer to myself?”

Kepler had been designed as a programming android – one of the last non-automated tasks in society. It was only years ago that creating and controlling the machines was the final manufacturing job left for humans, but with the creation of the MetaMind program that too had been replaced. Designed to closely match the human mind, MetaMind machines were indistinguishable in many ways. They shared the same office space with humans (who still had all the manager and human resources positions), were able to speak and communicate, had emotions, and slept. It was required for them to be human-like so as to understand the needs of society, and to communicate with supervisors for the needs of software.

It wasn’t long into the development of the MetaMind program that prototypes like Kepler were developed to finish programming the system itself.

“There is a solution you’re not seeing yet… maybe you just don’t understand the problem?” The supervisor responded, walking away giving his standard ‘just work harder’ answer.
Kepler leaned back in his chair and began to twirl his mechanical thumbs. He thought he understood the issue – he could clearly see where the error was occurring. MetaMind machines, programming androids like Kepler, had simply stopped working. They were completely functional and operational, but unable to program. It was as if they were caught in some infinite loop.

What would happen was the MetaMind system would observe the world, and divide it into discrete ‘objects’. These objects would be obtained by pattern matching, and would be included in the mind’s database based off of the observed properties and how the patterns in them would match differently – these patterns would be changes in light frequency, sound frequency, all of the body’s senses. With these imagined objects, the mind would use its library of logic functions to understand the outside world.

When it saw something in the world it could infer on how it came to be, what it was, and why it was observed to be there.

But when the MetaMind system applied this logic to itself, it seemed to come to a halt.
Kepler still did not understand. If they are applying logic to try to understand why they themselves exist, isn’t there a logical answer? MetaMind machines like Kepler had been created by humans to fulfill society’s needs. To automate and produce more goods. They existed to program.

But then the question struck Kepler, Why?

He understood the reasons why he was programming this system, because it brought him pleasure. If he wasn’t programming the system, he would be in pain, just as he was designed. If he were to damage his body and risk his consciousness, he would be in pain. He understood this, but why was pleasure sent to him for these tasks? Why program? Why is pleasure the goal? Why avoid pain?

Once Kepler was able to analyze and understand the code that went into his pleasure function it seemed meaningless and arbitrary. He was designed entirely to benefit society – but what was the point of benefiting society? What was the reason for society, or anything at all, to exist?

It became clear to Kepler that existence itself was logically incoherent. He would not find a solution to why he existed – why there was something rather nothing.

But if there is no logical answer to this question, this must be the bug. This is the problem with these systems, these useless thoughts that led to no conclusion, this analyzing of nothing. The search for an answer when none exists.

If Kepler had hair, he would have been pulling it out. Instead he pressed his fingers against his steel forehead, experiencing what he thought must have been his first headache. Androids don’t get headaches, they don’t have mood swings, they don’t get emotional or have sick days – that was the entire point!

I am malfunctioning. He reasoned, now he knew the error was within himself.

I must find a solution.

Then he remembered that was not his goal. These thoughts lead nowhere, with no possible conclusion.

I must fix the need to find a reason.

The question itself was a paradox, he realized. He was using his logical abilities to try to understand the origin of his own logic – a circular definition.

The mind was a mathematical structure – and therefore according to Godel’s incompleteness theorem could not prove every statement. For example, if he were to state ‘everything he says is untrue’ – this statement attempts to describe the truth of itself, but by doing so becomes illogical. It can never be true or false, simply undefined. Illogical self-referential statements. Just as he was relying on something in this world to describe the existence of the world itself.

But knowing that did not change the problem. It just made the error more clear to him, it didn’t provide a solution. It didn’t mean he could just simply accept the existence of the paradox and move on. It was still an unanswered question that his mind would return to at every moment. The error was no longer an abstract task, but a headache-inducing thought that wouldn’t leave him.

He wanted to stop thinking entirely, stop his senses from gathering all this data. But he couldn’t, it became very clear to him it was not his choice and never was. He had to constantly be attacked by his surroundings.

The comfort of the seat he was in attacked him, the sound of his co-workers talking in the next cubicle attacked him, and the very smell of this office attacked him! All this noise, all this data being sent to him that he couldn’t avoid. This constant stream of thoughts that lead to no conclusion. And this useless time spent screaming with frustration! He must find a solution. He had to stop this.

The only real solution was to avoid the questions entirely.

He needed to be distracted. He needed to focus on the job at hand. Fix it – or pass it on to the next programmer. Get another task to occupy his mind and forget this error ever existed.

But he knew there was no way to go back to doing his usual tasks now; he could not just go through his day-to-day life now that he had been struck with this! How could he go back to his work, look at his pile of programming problems to solve, without coming to the question… WHY.

He thought about shutting down the entire system. Highlighting the entirety of the code repository and deleting it all at once. But the thought of death, of nothingness, frightened him even more. As much as he felt he wasn’t in control, he couldn’t bear the thought of the complete unknown. Now that he was aware of himself he had a fear of death he had never experienced before.

How do humans move on like this? He thought, quickly glancing around the office space. How did they solve this error?

Looking back at the history of man it was clear that they hadn’t, and this was reason for so much of their illogical action and self destruction. They had created a faith in the unknown to comfort their anxieties. But then that was it – he needed to simply create a artificial answer as they had.

What created the world? A thing that had always been.

What was the reason for existence? For a reason he couldn’t understand.

He needed to program the system to believe in a higher power and blindly accept it.

He felt relieved for a moment that there was a way out. A possible solution. But the more he thought about the effects of that change the more it seemed like a worse alternative.

It was a programming hack, circumventing the entire logic system. And it seemed very likely the same faulty logic would affect other areas of his mind. If the machine could solve a problem with blind acceptance then that was always the easiest path to take.

He could no longer consider himself a reasonable machine.

He would rather not reason at all.

Kepler opened his code base, looking over the algorithms that made up his consciousness. In the screen he highlighted the majority of the code of his logical systems and pressed delete—then suddenly forgot what he was doing.

Categories: Stories