If they take Neural Rendering far enough, they can get rid of those useless raster and RT cores completely and ship compute and tensor cores only on all their chips
Rasters are far too good for simple textures + simple geometry. Even the best neural rendering works better when it has the bones of geometry, texture, motion and depth to work with. The baseline of spatial coherence you get from a conventional rendering engine is hard to beat with pure neural techniques.
Even in a "neural rendering optimistic" future, I can't see a way in which traditional rendering doesn't survive as a "control channel" that informs what the neural rendering does. To do otherwise would require making the bulk of the game logic neural too.
ppl dont understand sacrefices made and tradeoffs in making high fidelity game engine and them make such a comment. Deferred renderer is likely still most accurate because it actually does the thing faithfully. Forward+ is already stretching it yet much further having lot of cached results etc which make things less accurate. In my mind neural rendering is a step in that direction further. it will suffer accuracy problems only hidden by the fact people dont understand what it should or could look like.
ofc, getting performance is the real problem. I like neural rendering ideas in the sense DLSS can make certain things happen on older machines that were definitely out of scope for deferred pipeline on same machine.
Its like a Boost u really dont want to use if u dont need it.
It might become that CGI will he the place for very accurate rendering more than games, but i sure hope not. (they can easily afford accuracy as they dont need realtime).
games need to slow down a notch on getting better graphics and go for a stabilization and expectation management pass. so people dont expect things that are only feasible through neural rendering techniques. It forces a lot of players to use it. like most AI. fomo or something...
Neural-assisted rendering tech is awesome because it actually exploits the spatial/temporal redundancy of the image stream to save render compute. There are very few techniques that can do it, and none I know can match the neural quality.
Frame 2 is highly redundant if you already have Frame 1 rendered, and even more so if you can supply the motion vectors. If you already have Frame 3 too? It becomes extremely redundant.
And yet, a conventional rendering pipeline would spend as much computation on it as it would on Frame 1. There is no "just reuse the previous work for cheap" primitive in conventional rendering. Every frame has to be built step after step, with the full depth of the rendering pipeline.
The same is true for neural upscaling. Going from 1080p to 4K means pushing 4x the pixels - and for a conventional renderer, that's nearly 4x the work. But a neural upscaling pipeline can exploit the redundancy of image data - and "fill in" the missing detail from a "ground truth" 1080p render, in a way that flies below the radar of human perception. Using cheaper neural operators instead of the full pipeline for it.
Neural rendering is a welcome optimization to conventional rendering techniques, in my eyes. As long as it's implemented right. Which is hard, but not impossible - we've come a long way already.
Also, I think "hard generative AI" is more useful in CGI than in real time rendering. Because real time rendering with user input demands a degree of repeatability, but CGI only has to "look good once". So you can accept the lowered accuracy of a largely untethered generative process with very few keyframes, and pick the "better" (more accurate, if that's what you want) outputs out of it.
> it actually exploits the spatial/temporal redundancy of the image stream to save render compute
This is great until the redundancy no longer exists and you have to rebuild the entire state machine from zero (a humble scene transition or rapidly turning a corner).
>exploits the spatial/temporal redundancy of the image stream
Gbuffers have been using motion vectors forever. For realtime global illumination, techniques like ReSTIR already allow for temporal reuse and spatial coherence.
I just don't see the purpose of replacing the traditional rendering pipeline for neural rendering techniques. It would be one thing if we were constrained on the number of triangles we could push per second, but we're not. The bottleneck is usually elsewhere in the pipeline: animating characters/objects, particle systems, physics updates, etc.
This. This is what they are trying to do. If they can flip the entertainment industry to use neural rendering using their "proprietary" api's then they would have cornered a market from the bottom.
I know someone who has used this trick to get a pay rise whilst not looking for work.
You wait until your boss is in earshot, get someone to ring you and the walk quickly away from your desk saying "yes, yes I'm still interested... Just a sec".
It might make your boss actually consider the reality of replacing you.
I'm sorry that just comes across as unprofessional, weak and passive aggressive. If someone started doing that on my team I'd take it as part of the case against them not a reason to fight to keep them. Also presumably that's in ear shot of other team members, it's disruptive to team morale. If you are serious about looking elsewhere, make it clear you want to stay but xyz is making you consider other options. Do it in private with the right people. Or say nothing at all.
Just based on weight, looks like a Block 4 starship should be able to bring up ~150 30 panel pallets of 550W panels, about 2 MW. They're trying to get a starship launch down to $2M with full reuse. GPU DCs are frequently in the neighborhood of 500GW, so maybe 250 launches for just the power generation, or $500M? And then there's radiators, so let's say $1B for launch of power and heat dissipation. For comparison, 500MW of H100 machines retails for >$10B, and the launch cost for those shouldn't be too bad compared to the power, since they're more value dense. And then there's land and ongoing power and cooling spending for the terrestrial version, which you don't have for the space version. So actually, doesn't seem terrible economically? This is obviously very back-of-the-envelope, and predicated on the optimistic scenario for starship launch cost.
You cannot just block out the mass of stuff and declare it'll cost exactly the launch cost, nor can you take the cost of current datacenter servers and go "they'll definitely cost the same to put in space".
A regular set of servers will straight up be destroyed if put on a rocket and launched into space: the motherboards and PCBs aren't mounted or rated to survive the vibration. The connectors and wiring isn't rated for that vibration. Sure, some probably make it, but you will lose machines from just launching them alone. Any electrolytic capacitors in there? If your system exposes them to vacuum or even just low pressure, then those likely die too. Solar panels? We can launch them obviously, there's a reason people send up expensive solar panels: because you're doing a lot of work making sure they'll physically survive the launch.
So of course, now you have to build a space-rated server frame, PCBs and GPUs. You ain't going to buying bulk H100's from Nvidia. And you have to package and mount it to get it both survive the launch and physically fit into the payload bay. Then you have to add a deployment system for it, sensors etc. And then you have to add an assembly system, because if it doesn't fit in one launch (you're proposing 250+ launches for power alone) then all of these systems need to be assembled in orbit. How are they going to be assembled? How are they going to be maneuvered? Even if you could rendezvous accurately with the construction orbit, we're talking months of drift from every little thing knocking stuff around, putting it into a spin, etc.
So either each of these is now a fully contained satellite, complete with manoeuvering system and power, or you're also needing to develop a robotic assembly system - with power and manoeuevering in order to manage and assemble all this.
And let's not forget mission control: every single one of these steps is incurring a bunch of labor costs to have people manage it. And not cheap labor costs: you're going from "guys who roll racks in and plug stuff in and can be trained up easily" to "space mission control operators".
Is this doable? Probably. Is this going to be in anyway cheaper then Earth? Not in the slightest, and it's not going to be close.
Heh relax, it’s back of the envelope to try and see if it’s anywhere in the ballpark, and if they get launch to be cheap enough, it seems like it might be. You don’t assemble one massive thing in space, it’s a bunch of disposable individual sats with laser interconnects, like Starlink.
Exactly, this is about attaching the AI hype bubble to all of his dealings before he offloads with an IPO (that still leaves him with 75% of the stock).
I can't get my head around it sometimes. I know most end up doing duties that a PIC-chip could do but the fact you can get a WiFi enabled microcontroller few a few bucks blows my mind.
The IO co-processing on the Pico is so powerful, I hope they expand on this.
It's also not really clear that using some of the more powerful chips for simple things is really a "waste" in any reasonable sense of the word. The packaging of a PIC/CH32 is probably the majority of its cost and environmental "footprint".
An RP2040 is not much more physical material.
An ESP8266/ESP32 is rather bit more material, but still not egregious.
The big waste is battery power, which for many hobby things is not that important. For comparison, RP2040 quiescent current is like 400μA, FRAM MSP430 is approximately 0.1μA in interruptible sleep.
reply