Problems with Distributed Computing

[julps31]

Exactly. Every time I think how can that work? I think to myself, theres been people thinking the same thing for almost five years and those people know what they're doing. And they do it as a full time job. So i'm sure they can get around any problem with the amount of people they have and the amount of time they have.

[cpiasminc]

Personally, my faith lies not so much in the fact that they have 300+ engineers working on Cell, but in the fact that they're declaring war on DirectX and XNA. The fact that Criterion is pouring so much into making Renderware 4 easy for developers. The fact that there were so many complaints about Sony's early-stage documentation on PS2.

The PS1 documentation was so-so, but the architecture was simple and straightforward enough that people could easily figure it out. PS2's architecture was a lot more exotic, and people were expecting something similar to the PS1 -- what they got was closer to the Saturn. And the initial documentation was just plain awful. That's why there's such a huge graphical difference from the early generation to the later generations. After years of learning more about the platform, games went from 40,000 polys per frame to 150,000 polys per frame on the same hardware. PS3 is kind of a mix, I'd say. The relationship between devices and the limitations imposed are not that unusual per se, and the fact that they all have a unified ISA is most encouraging. But there's just so friggin' many computational resources that it'd be a pain to singlehandedly manage them all.

I do think that this whole "software cells" type thing will play a big role in how easily the multithreading and distributed computing is handled. But nothing is really made clear what that means to a developer. No matter what, though, someone's going to have some complaints about it. Don't get me wrong, though -- I don't expect any better from Microsoft or Nintendo, either.

[welcome_to_the_3rd_place]

Now I know why I skipped the last topic I replied to

YAWWWWNNN!

Lets just say this.

Sony would not release this machine then after release go "oh f**k - it doesn't work after all - our games are running crap on every machine!!!"

You've al got ahead of yourselves and forgot one thing.

A PS3 not connected to any other cell based machined - LAN/WAN or internet (also have you any idea how fast a LAN can be these days??? we are talking precisely 0ms of lag for LAN technology that exisits now) will run PS3 games absolutely fine.

Don't get all excited about this technology - the cell will be ahead of the surrounding technology but in 3 years time, if you don't have at least a 1Mbps connection to the internet (megabit that is) you're in the dark ages and shouldn't have technology.

[cpiasminc]

A PS3 not connected to any other cell based machined - LAN/WAN or internet (also have you any idea how fast a LAN can be these days??? we are talking precisely 0ms of lag for LAN technology that exisits now) will run PS3 games absolutely fine.

Not precisely 0ms. That's physically impossible. when your ping times are "0", that means "<= 0.5 ms" (round-to-even means that 0.5 will not round up to 1). Either way, as I pointed out, Deadmeat made erroneous calculations that 60 fps means a 1.6 ms update. He was off by an order of magnitude -- it's 16 ms. Either way, I agree with you as far as distributed processor sharing over a LAN. Limited possibilities over a WAN... but over Internet, absolutely not. There, you'll easily have lags in dozens and dozens of ms.

Don't get all excited about this technology - the cell will be ahead of the surrounding technology but in 3 years time, if you don't have at least a 1Mbps connection to the internet (megabit that is) you're in the dark ages and shouldn't have technology.

1 Mbit? That's actually not fast at all. For that matter, you make it sound like people always have a choice. It's foolish to blame people for living in a place where broadband isn't available. Cable modem connections are generally 3 Mbps nowadays in the U.Ss, but that is a recent upgrade which probably hasn't proliferated to every customer. DSL has not sped up one bit over several years, and there are plenty of places throughout the world or each country where DSL is your only option for broadband. Or for that matter, in most cases, the cost difference per month between cable and DSL is too high to even consider cable.

Also, if you've ever lived in a college dorm, you'll know that you have ridiculously fast connections (usually 100Base LAN pipe into an optical carrier line), but you'll have a limited bandwidth usage per month. In my school, you were limited to 500 MB per month... which if you toy with multiplayer gaming over the internet, that's damn near nothing.

[imported_The_One]

Don't get all excited about this technology - the cell will be ahead of the surrounding technology but in 3 years time, if you don't have at least a 1Mbps connection to the internet (megabit that is) you're in the dark ages and shouldn't have technology.

Dude, I EASILY have a 1MB (actually, 1.25MB, yes, bytes) connection for a mere 30 CDN per month. 1MB will be nothing in 3 years... Oh yes, if you're wondering, I have a 10Mb Cable connectiong, and this isn't even the fastest service (It's fast... but it easily eats up your bandwidth, stupid ISP only give you 4GB of bandwidth per month... Hell, that's LOW). However, ADSL (Asymetric Digital Subscriber Line... What's the difference between ADSL and DSL anyways?) in my area has a maximum speed of 2.5Mb (And they give ADSL 6GB of bandwidth... Unfair!!:roll.

but in the fact that they're declaring war on DirectX and XNA.

Mind elaborating on this point? I personally see no significant value declaring war on DirectX and XNA will have on distributive computing.

[cpiasminc]

Mind elaborating on this point? I personally see no significant value declaring war on DirectX and XNA will have on distributive computing.

Not Distributed computing per se. I was referring to the point of making PS3 development easy. DirectXNext, but more so, XNA are touted as things that will make technology development easy and leave more time for development of the game. If Sony is declaring war on XNA, that suggests some commitment towards making some really robust and powerful development tools.

[imported_The_One]

Ah yes, I see your point there.

[cpiasminc]

Dude, I EASILY have a 1MB (actually, 1.25MB, yes, bytes) connection for a mere 30 CDN per month. 1MB will be nothing in 3 years... Oh yes, if you're wondering, I have a 10Mb Cable connectiong, and this isn't even the fastest service

AAAAhhh!! GIMME! That's frickin' unfair. I can't even get a *regular* phone line for $30 CDN ($22.75 USD)! The fastest you can get in this country for a livable price is 3 Mbit/sec w/ 512k/sec upload. Although there's no bandwidth limit, it costs me twice as much (about $45 USD/59 CDN per month).

ADSL max speed only gets up to half that, and there's no guarantee that you'll ever actually reach the full bandwidth -- if you're near the distance limit, you'll likely never go above 75% of the actual bandwidth (noise issues).

BTW, ADSL specifically refers to "assymetric" DSL : Different upload and download rates. Often times, the same providers will also offer corporate DSL, which will actually be symmetric (SDSL) : same upload and download. There's also a handful of places where corporate buyers can get VDSL (very-high bitrate DSL), which offers speeds as high as 50 Mbits. Unfortunately, the key to VDSL is that its second stage (right after copper) is fiber-optic, so the provider needs to have a full-fiber network. And the other problem is that it can only maintain its bandwidth over about 4000 ft of copper line. So it's extraordinarily expensive. About as costly as a T3 line.... unless of course you live in Hong Kong (or was it Taiwan?)... where 55 Mbits/sec is status quo, and it costs the equivalent of $30 USD a month. :x :x :x

EFM (802.3ah), where are you?!??!???

[imported_The_One]

I never found out... How much does T3 cost? And what's the upstream and downstream bitrate?

[cpiasminc]

T3, last I recall costs a few dollars (maybe $2.00 - $2.50) per hour. Which really adds up considering you have a 24-7-365 connection. T3 speeds are symmetric, 45 Mbits both ways.

Of course, VDSL is still cheaper than T3 for setup costs. The unit is still an ordinary DSL modem that operates at a much higher frequency. T3, otoh, uses a very unique router that can easily cost upwards of $8,000, and you have to pay it because there's no other alternative.

EFM is a far better solution for the future. It uses the existing copper lines and gets anywhere from 10 to 75 Mbits/sec (depending on line quality) and it's a total switching upgrade, so once the phone company adopts it, everybody has it. The problem is that you can't restrict it to individual customers -- it upgrades entire towns at once (which is something the phone company doesn't like since they make less money that way).

[5ysT3m cR45h3r]

In some ways, the PS3 may create a digital mainframe similar to that of the Matrix, but without people being controlled by machines that have enslaved humanity.

[Ibanez32]

Perhaps if sony want to impliment this it won't be available on release. Well definitly not in the uk any how.

[5ysT3m cR45h3r]

It would be nice if this was implemented on release, no doubt. I guess we'll just have to keep guessing until it's official.

[MegaGrid]

I have yet to see such a beautiful thread describing the distributed nature of the PS3. Of course 1TFlop is a nice processing power but the distributed nature is what will make this console grand I agree.
I would quite agree that the general concept of the distributed nature of PS3 sounds a little bit communistic, many things sound like this but end up quite profitable as I suspect it will be in this case, making the PS3 distributed nature spread in all colours of the political prism
Of course lets not forget Napster and many other distributed sources.
Many people have benefited from this.
Whats the difference? In Napster things are offline gaining sth when you are not in your Computer...
Anyway this is why I suspect two things to happen in the routing and the user level.
The routing level is being touched by QOS and well explained.
Now in user level, user rings in companies, in groups will be created that will give privilages to its members and only to those, this also steps into the QOS.
So I belive that QOS is a bullseye to the scenario of the distributed nature of PS3.
Now Sony has invested $500 million, I do not believe that these chips are only going to stay inside Sony, Toshiba and IBM... for some years it might.
Anyway I love the nature of the Cell processor which gives it sth like a virtual presense in distributed programming. Very very nice.
Also I love this thread and in genral forum and congratultions to the Mods.
Very very nice base to the upcoming PS3.

Alex.

[Xeno]

Sony has invested far more than $500 million into CELL...

Distributed computing would be better suited for a tech savy area like Japan where everyone and their 3rd cousin have insane internet speeds.America and Europe are in the stone age when it comes to the internet. ops:

[imported_The_One]

Sony has invested far more than $500 million into CELL...

Distributed computing would be better suited for a tech savy area like Japan where everyone and their 3rd cousin have insane internet speeds.America and Europe are in the stone age when it comes to the internet. ops:

I know how you feel... still stuck with 3Mb connections XD :roll:.

[MegaGrid]

Distributed Rendering in the pixel level guys is not a good idea guys. I liked your general concepts thoughts.
There are a lot of stages before going to the pixel level...
Anyway, sending data before needed... and come back when needed how?
With QOS, as I said QOS is a necessity. Even though presending in a statistical sense, or sending scenes to be rendered before rendered, with the physics and AI resolved sounds great dont you think?
As for the GPU (Graphics Synthesizer) who knows... They have to do hard work on this. Now as we know a nice NVIDIA GPU costs now $300 and more... which is near the expected price of the PS3... ($400-500) and these GPUs need a lot of juice to work...
Do not even say a real time distributed rendering.
As I said it will work and it will be fantastic.
Think of it as the bulk of the Data stream reuniting at the proper time.

Alexander.

[MegaGrid]

You have to think about the concept of the Multimedia Data Stream , now it is inside the cell.
Anyway I imagine the PS3 in my house as my main horse power in rendering to the pixel level and processed data streams (cells) flowing constantly from mine or other PUs to the GS (or how they will call it). I can not imagine anything else and I do not dare for the next decade.
The last 100meters will be run by the machine in my house.

Alex.

[MegaGrid]

I would agree with the cell naming... it is like the blood in our system.
You need good vessels for this...
So in the computer paradigm the instructions and data inside the cell do not get blocked. I would put my money not so much in the speed for the time being but in QOS. Making the cells flow in a scheduled (or close to) manner.

Alex.

[cpiasminc]

Distributed Rendering in the pixel level guys is not a good idea guys. I liked your general concepts thoughts. There are a lot of stages before going to the pixel level...

Well... not unless you're trying to do raytracing... :wink:
When you're raytracing, the pixel level is the first stage.

But yeah, realtime distributed *rendering* is out of the question. 60 fps means 16 ms to be sending out and recovering some significant chunk of the image. That's not likely to help you much at all even in a LAN.

Distributed computing can do a hell of a lot for things like massively parallel AI or physics or various game logic miscellany. Especially things that don't necessarily need to be updated on every frame or calculate so fast that you're not going to suffer more than a few ms of delay by delegating them to other machines.

Anyway, sending data before needed... and come back when needed how?

Well, according to the whole "software cells" idea, the chunks of code to be processed as cells come in to the CPU as IP packets. Being IP packets means you know what IP they came from, so it should be a pretty safe assumption that if a codeblock came from some machine A, the same machine would want the output of that codeblock, now wouldn't it?

I agree, though, that quality of service is a key point, which is why it's not realistic to see any sort of distributed computing in realtime outside of a LAN. I don't care if it's PCs or PS3s or supercomputing clusters.