Mythruna
March 28, 2024, 08:44:09 AM *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
News: Welcome to the new forums. See "Announcements" for a note for new users.
 
   Home   Help Search Login Register  
Pages: [1] 2
  Print  
Author Topic: Overcoming "fear of failure"...  (Read 28265 times)
pspeed
Administrator
Hero Member
*****
Posts: 5612



View Profile
« on: June 01, 2015, 12:38:19 AM »

First, to catch folks up who might just be joining...

What seems recent but has been a couple years now, I wrote a high-performance network object synching library for Mythruna... in prototype form.  It's zone based, has internal delta compression, and so on.  All very fancy.

This past year, I got the idea to open source this library for several reasons.  It will force me to clean up the design and decouple it from some Mythruna specific things.  It will also expose its bugs/fixes/patches to a wider audience.  The clever parts of this code are just about at the limits of my cleverness which means I'm probably not able to debug it properly on my own anyway (according to conventional wisdom).  At any rate, I use it in a specific way.  Others will bend it in different ways and discover bugs under rocks I never overturned.

Everyone wins.

In that light, for the past few weeks, I've been recoding this library one bit at a time... testing what I could when I could.  It's been a long road.

This open source library will be called Sim-Ethereal.  That should catch everyone up.

Fear of Failure

I suffer from a kind of subtle fear of failure sometimes that can be insidious.  Perfectionism is one common root of procrastination and the 'fear of failure' can be one of the gateway links.  When time and motivation are already balancing on the head of a pin, any little upset can have far reaching consequences and so I try to find strategies to overcome it.

A few factors contribute to this effect for me and a particular project:
1) is this an important thing for which failure ruins a bunch of other stuff,
2) is it a job that's big enough that it's not easy to see the whole thing all at once,
3) and is it complicated enough that there will be numerous major and fundamental design crossroads for which turning back will be expensive.

Any of these on their own can be demotivational but for various reasons, in me, the combination is deadly.

Sim-Ethereal definitely falls into this category.  It's a fundamental missing building block in almost every game project idea I ever conceive of.  Furthermore, without it, a bunch of Mythruna's planned features vanish in a puff of smoke.  The job has stretched over the better part of a month and will continue to do so.  The delta compression over unreliable linkages has stretched the limits of my brain's internal state tracking and there have been several major design decisions with far-reaching ramifications.  It's got it all.

Now, first, let me remind the humble reader that a "fear of failure" is not always logical.  A networking library is not an impossible thing.  Game developers much less experienced than I am have come up with perfectly functional networking libraries.  My requirements may be kind of extreme but they are no more onerous than Valve's or any other AAA studio's... and as they say, it's not exactly rocket science.

It might be a big job for one guy with lots of life issues tugging at his fringes but it's not an impossible job.  It just isn't.  Even a crappy or partial implementation would probably be ok, even.

Anyway, this feedback loop has the potential to be deadly to a project:
-hit "yet another design issue"
-day dream a bit about possible solutions
-these thoughts lead to reminders of the ancillary requirements of the 50 other projects you have in mind for this library
-this leads to thinking about all of the "cool stuff" you'll be able to do when the project has finished... and possibly even more stuff.
-meanwhile, the design decisions has gotten no easier, the project has gotten no smaller, and no progress has been made.

At least half the time, thinking about the problem may lead to more discovered issues.  "Aw man, this assumption I made last week is wrong... isn't it?  Maybe, yes?  No.  Well, yes..."  Then I sleep on it, fitfully, dream about horrible things, and hopefully have some new ideas in the morning.

Rinse, repeat.

Wrestling with a Dilema

Truthfully, some problems just plain require worrying about them.  There might be a logical way to divide and conquer or some other way to systematically approach a solution but way way way too often, a problem just requires endlessly gnawing at it until a solution presents itself.  I can't count the number of times Mythruna-related development has worked this way.  

For days there will seem to be no way in until an idea blossoms.  It's almost never a lightening bolt.  It will be some thing I didn't notice.  Some way of looking at an issue from a different angle, or masking out some part that was distracting me, or... too many things to describe.  Slowly, the knot comes untied and the solution is so clear that it almost seems obvious.

Rational me understands this but fear-of-failure me needs constant reminding.  If it's been a while since I've experienced this, it is easy for the fear of failure to whisper in my ear "This is harder than those other problems you barely remember... there may be no solution this time... look at the calendar... how much time are you willing to waste on this approach..."

Even as a write this, there is a little nagging voice telling me that stepping away from a problem is for quitters.

Doing the Math

Many years ago, I put in a backyard playground for my kids.  It's not large, but large enough.  With slop, I calculated that I'd need something like 14 cubic yards of mulch to create the recommended depth, etc..  (I'll admit I could be remembering the numbers wrong but the rest of the story will put it in perspective if the numbers are off.)  So I found a place that would deliver playground mulch and sometime later a dump truck dropped a full load of mulch at the foot of my driveway.

A dump truck can hold a lot of mulch.  I mean a lot.  This pile was very big.  Seeing my little wheel barrow sitting next to it was depressing.

To get the mulch back to where it needed to be was lots of 'fun'.  If you imagine my lot as a big squarish area, the pile of mulch was on the opposite corner from where it needed to be... and obviously there is a big house in the way.  We'd load a wheel barrow full with snow shovels and then wheel it back over uneven ground, through the gate, around the house (and pool) and to where it needed to go: plop... then repeat the process.

Then back to the front to see a pile that looked exactly as big as it had before.

It took all day and into the evening to move that pile.  By the end, we were working from flood lights just to get the job done.

For fully half of that time, it felt like we'd never finish.  We might not have been moving a mountain of sand with a spoon but it certainly felt like a job more similar to that than anything I'd done previously.  (People who do or have done this work for a living are welcome to laugh at me... but I doubt you'll tell me it's easy work.)

Concentrating on the math was what saved me.  The irrational part of my brain only saw more mulch than it could comprehend.  It wanted to quit... with every trip.  The rational part of my brain knew how much mulch I ordered and how much a single wheel barrow would contain.  It focused on a rough estimate of the number of trips and tried to keep a rough tally of how many trips were left.  As I recall, I pegged it within +5/6 trips or so... but by then the pile was down to a size where even my irrational side has to shut up.

Conclusion

I cannot tell you how often I kept that day in my mind these past few weeks.  When life was pulling at me from five different directions and the amount of time I could spend on Sim-Ethereal was pulled apart, I kept thinking of that relatively little wheel barrow moving mulch from a pile the size of a dump truck bed.  I kept thinking if I could even spend 20 minutes on it then at least I was still keeping the issues fresh in my brain.

But mostly it was a case where rational thought won.  I can point to that.  The feeling of the size of that pile is still viscerally real to me but we moved it just the same.

Fast forward to this weekend.

Despite another two major design redirections (both of which improved the library greatly, mind you), I was able to make enough progress that I'm seeing object state messages on the client... and they have the right values.  Given the layers that information has to travel through, you'll have to trust me that it's a pretty significant milestone.  The pile is now below the fear-of-failure threshold.  Rational thought can take over.  It is entirely plausible that all major design decisions for the library have been handled (never say never but I'm optimistic).

There is still plenty to do but now it's in the realm of just hooking it all up.  It may take another week before I have it fully integrated into the E-space test game... depending on day-job obligations.

It may be more than a month later than I wanted to be right now... but the fear-of-failure can suck it.  Finishing late is still finishing.
Logged
Rayblon
Donators
Hero Member
***
Posts: 1861


Hmmm...


View Profile
« Reply #1 on: June 01, 2015, 03:04:24 AM »

Procrastination aside, I think your 'subtle' fear has helped you create things that may one day rival will surpass AAA titles and content, if only in their quality. I remember one of the biggest hurdles for 1.8 Minecraft was the fact that they had to recode the entire engine because it was so... bad. The ended up having to change over 1,000,000 lines of code because they did a sloppy job at first. The way I see it, after you have your way with something, it's platinum. Even if you overlook something, your content still rivals puts to shame what the millionaire studios are dishing out.
« Last Edit: June 01, 2015, 03:27:28 AM by Rayblon » Logged

Michael
Donators
Hero Member
***
Posts: 2166



View Profile
« Reply #2 on: June 01, 2015, 11:57:41 AM »

Thinking about that pile of mulch made me want to have irrational thoughts about ever finishing, too..

I remember that Rayblon, and that makes me think that once I learn what I'm learning I'll end up rewriting my whole project (or bit by bit right now)..
Logged
Rayblon
Donators
Hero Member
***
Posts: 1861


Hmmm...


View Profile
« Reply #3 on: June 01, 2015, 12:43:52 PM »

Thinking about that pile of mulch made me want to have irrational thoughts about ever finishing, too..

I've never had this issue. I just mentally groan at how many months a project may take.

Funny thing is, most of the projects I've started will never be finished.
Logged

pspeed
Administrator
Hero Member
*****
Posts: 5612



View Profile
« Reply #4 on: June 01, 2015, 02:23:01 PM »

Thinking about that pile of mulch made me want to have irrational thoughts about ever finishing, too..

I've never had this issue. I just mentally groan at how many months a project may take.

Funny thing is, most of the projects I've started will never be finished.

Yes, and that's what the 'fear of failure' will be constantly reminding you: "This is just another one of 'those' projects.  It will be among good company with all of the other cool stuff you never finished..."
Logged
ebag51
Donators
Hero Member
***
Posts: 531


View Profile
« Reply #5 on: June 01, 2015, 02:28:57 PM »

I still have faith in Mythruna Paul, and I will continue to wait for it for years to come(Give or take a few periods of inactivity on the forums).
Logged

Rayblon
Donators
Hero Member
***
Posts: 1861


Hmmm...


View Profile
« Reply #6 on: June 01, 2015, 02:53:24 PM »

Thinking about that pile of mulch made me want to have irrational thoughts about ever finishing, too..

I've never had this issue. I just mentally groan at how many months a project may take.

Funny thing is, most of the projects I've started will never be finished.

Yes, and that's what the 'fear of failure' will be constantly reminding you: "This is just another one of 'those' projects.  It will be among good company with all of the other cool stuff you never finished..."

None of my Mythruna projects have been abandoned, though. :>
Logged

Rayblon
Donators
Hero Member
***
Posts: 1861


Hmmm...


View Profile
« Reply #7 on: June 01, 2015, 02:55:18 PM »

Give or take a few periods of inactivity on the forums



NOBODY CAN TAKE A BREAK FROM THE CULT OF EVIL LAUGHTER.
« Last Edit: June 01, 2015, 02:57:28 PM by Rayblon » Logged

Rayblon
Donators
Hero Member
***
Posts: 1861


Hmmm...


View Profile
« Reply #8 on: June 01, 2015, 08:15:03 PM »

http://blog.roblox.com/2015/06/create-all-new-worlds-with-smooth-terrain/

Normally I wouldn't give any praise to this company but... Oh man. Bravo. They did the impossible and it looks freaking gorgeous.

Kind of like what you will do.
Logged

pspeed
Administrator
Hero Member
*****
Posts: 5612



View Profile
« Reply #9 on: June 02, 2015, 07:10:58 AM »

Last night I had a little time after finishing "day job" stuff (more like giving up on a 'day job' problem for the day) and so looked at hooking  up the messages needed to get the delta compression working.

The old function/method that updated the object state baseline was a couple pages long.  So I was preparing myself for the conversion to take some time... but because the new design is nicer and the lower level parts were already implemented well (and better organized), the new code was only 4-5 lines.  Basically: "loop over objects, call their updateBaseline method".

Took more time to make sure that the server was receiving the acknowledge messages from the client than it did to hook them up... and  hooking them up automatically hooked up the client, too.

It's nice to have an easy win and makes me feel better about the design at the same time.  Much better than the previous night where I spent an hour tracking down an issue that was a type in deeply low level code... a cut-paste error where a 'y' was an 'x'.  Oops.  Those kinds of nights undermine confidence.  Nice to have a good one to balance it.
Logged
pspeed
Administrator
Hero Member
*****
Posts: 5612



View Profile
« Reply #10 on: June 06, 2015, 01:34:14 AM »

This week I've done my best to try to chip away at this a little a day.  Work has been pretty crazy but I've tried to keep to a rule of working on ethereal at least every night even if it's only for 15 minutes.  Usually it amounts to more... sometimes it's just a matter of getting started.

So tonight I finally have my space ship flying around over the network and I was able to log two clients in and see.  Yay.  There are lots of bugs, of course (boo) but that's to be somewhat expected at this stage.

From the client on the same machine as the server, everything is super smooth.  There are some glitches with a delay in object removal causing the lasers to pause at the end of their fly time... but I kind of expected those sorts of issues.

Also, on the client that is not local to the server, the movement is very jerky much of the time.  And it's not just that ship's movement but all movement so I have some things to look into there.  It could just be that my history window is not large enough.

The more trouble issue is that I occasionally see objects in the wrong zone.  So there is some issue with zone homing and/or overlapping zones or something.  In one case, it didn't fix itself until I flew away and back again.

On the plus side, these are the sorts of issues that are "real" and I knew that I'd be debugging as soon as I got the library to this point.  So it's still part of the progress... I was just supposed to be here a month and a half ago.

Oh well, progress is progress.
Logged
Rayblon
Donators
Hero Member
***
Posts: 1861


Hmmm...


View Profile
« Reply #11 on: June 06, 2015, 04:32:27 AM »

Oh well, progress is progress.

When do you think a playable release of Ethreal Space will be ready?  Cool
Logged

pspeed
Administrator
Hero Member
*****
Posts: 5612



View Profile
« Reply #12 on: June 06, 2015, 12:52:45 PM »

Oh well, progress is progress.

When do you think a playable release of Ethreal Space will be ready?  Cool

I have no idea.  Depends on how long it takes for me to track the bugs and stuff.
Logged
pspeed
Administrator
Hero Member
*****
Posts: 5612



View Profile
« Reply #13 on: June 08, 2015, 12:55:04 AM »

I've been taking my time tracking down the zone bug because I feel like building out the infrastructure for tracking down problems is apropos at this point.  Networking bugs are notoriously hard to find so whatever tools I can give myself are gold nuggets.  In fact, just creating the debug layers has already pointed to one problem I thought I had (and turned out to be something else entirely).

Here is a picture of the layers I've added so far:


In the upper corner, we see this player's current location and zone. (The font's Z is kind of funny... but whatever.)  On screen, we see the local zones around the player.  These are fixed in space and there is always that 3x3 grid around the player.  As we cross one border, it adjusts to always keep a one zone border around us.  My zones are too small for the real game but good for debugging... but that's why I thought the lasers had a problem pausing in space... once an object leaves the watched zones it just freezes.  No more updates.

Over each mobile object with some 'name' text, I also get its name and its current zone.  This will be useful for tracking down the cases where I was seeing an object appear where it shouldn't be... because that indicates that the network layer is giving me the wrong zone.

Right now, this zone is based on the position is the client detects it... which on its own won't tell me anything, really.  Next step will be adding some additional debug info that includes the zone as the server sees it.  Then we'll be getting somewhere.  Not only can I then display both, but I can indicate errors when they are different and stuff.  I can even add information from each stage of the process: zone as the server sees it, zone as the network client received it, actual displayed zone based on client-side world position.

The other plus side is as I've been thinking about this, I think I might have some clues as to the issue.  So letting is simmer hasn't been bad either.

Whatever I can do to find a way to test this without logging two clients in would be best.  And that's what I'm trying to get to... narrow the problem down to the point where I can cause it with a moving object that isn't controller from another computer... because testing with two clients takes way longer. 

I've even considered adding some simple AI ships just to fly some patterns to test various theories... and I might still do it.  That has double-benefits.  But if I can solve this zone problem without it then I will.
Logged
pspeed
Administrator
Hero Member
*****
Posts: 5612



View Profile
« Reply #14 on: June 08, 2015, 11:18:47 PM »

So, after adding all of this cool debug info... turns out that the zone 'bug' I saw before might not have really been a bug at all but my misinterpreting something else I'd left out.

...basically the same problem as the lasers 'pausing' except in this case the objects were still live.  They'd gone outside of my normal zone view but didn't disappear.  So I was just seeing the other ship hanging out in space somewhere but it wasn't the real ship.  The real ship had flown to a zone I couldn't see. 

Once I'd started adding the zone information as the server saw it, I could easily see that the object was still moving around, just somewhere else.  So the debug layers really did end up saving me a lot of trouble because it turns out I'd completely misunderstood what I was seeing.

I just need to make objects disappear once they are 'out of view'... ie: once we stop getting updates about them.

On the plus side, I flew around following an asteroid for quite a while without any major issues.  There were a couple places where it jumped because of network hiccups and I noticed that it's bounding radius is offset strangely but the rest is fairly encouraging.

Maybe by this weekend I'll be ready to see how this runs over the real internet.  (bites nails nervously)
Logged
Pages: [1] 2
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.20 | SMF © 2013, Simple Machines Valid XHTML 1.0! Valid CSS!