Friday, February 14, 2025

Latest Posts

NVIDIA provides instruments it says will speed up improvement of humanoid robots


Take heed to this text

Voiced by Amazon Polly
Developers can use Apple Vision Pro to teleoperate robots for capturing demonstration data, says NVIDIA.

Builders can use Apple Imaginative and prescient Professional to teleoperate robots for capturing demonstration information. Supply: NVIDIA

At SIGGRAPH in the present day in Denver, NVIDIA Corp. introduced analysis and choices for simulation, generative synthetic intelligence, and robotics. The corporate mentioned it’s offering a collection of companies, fashions, and computing platforms to allow robotics and AI designers “to develop, practice, and construct the subsequent era of humanoid robotics.”

“The subsequent wave of AI is robotics, and one of the crucial thrilling developments is humanoid robots,” acknowledged Jensen Huang, founder and CEO of NVIDIA. “We’re advancing all the NVIDIA robotics stack, opening entry for worldwide humanoid robotics builders and corporations to make use of the platforms, acceleration libraries, and AI fashions finest fitted to their wants.”

 The firm is presenting 20 analysis papers at SIGGRAPH. Rev Labaredian, vice chairman of Omniverse and virtualization technique, mentioned in a press briefing that NVIDIA has been engaged on graphics analysis since 2001, so its connection to graphics, simulation, and robotics is well-established.

NVIDIA NIMs assist develop digital twins

Not solely can simulation assist design robots and their environments, but it surely can be utilized to coaching manufacturing methods, mentioned NVIDIA. Its NIM microservices are pre-built containers utilizing NVIDIA inference software program, now provided as a service. The corporate claimed that they will cut back mannequin deployment instances from weeks to minutes.

“The time to use generative AI is now, however it may be daunting,” acknowledged Kari Briski, vice chairman of generative AI software program product administration at NVIDIA. “Enterprises want a quick path to manufacturing for return on funding. … This led to NVIDIA NIMs to standardize deployment of AI fashions, and it’s primarily based on CUDA to expire of the field.”

“Two new AI microservices will enable roboticists to reinforce simulation workflows for generative bodily AI in NVIDIA Isaac Sim, a reference utility for robotics simulation constructed on the NVIDIA Omniverse platform,” the corporate asserted.

The MimicGen NIM generates artificial movement information primarily based on recordings of teleoperation utilizing spatial computing gadgets akin to Apple Imaginative and prescient Professional.

The Robocasa NIM generates robotic duties and simulation-ready environments in NVIDIA’s OpenUSD Common Scene Description framework for growing and collaborating inside 3D worlds.

The corporate additionally introduced NIM microservices to help within the improvement of digital twins, in addition to USD connectors to permit customers to stream large NVIDIA RTX ray-traced information units to Apple Imaginative and prescient Professional.

The USD Search NIM for working with OpenUSD can perform text-based searches of 3D assets, says NVIDIA.

The USD Search NIM for working with OpenUSD can carry out text-based searches of 3D property. Supply: NVIDIA

OSMO permits robotic orchestration from the cloud

Accessible now, NVIDIA OSMO is a managed cloud service that robotics builders can use to orchestrate and scale multi-stage workflows throughout distributed computing assets, whether or not on premises or within the cloud.

“OSMO vastly simplifies robotic coaching and simulation workflows, reducing deployment and improvement cycle instances from months to underneath per week,” mentioned NVIDIA. “Customers can visualize and handle a spread of duties — like producing artificial information, coaching fashions, conducting reinforcement studying and implementing software-in-the-loop testing at scale for humanoids, autonomous cell robots [AMRs], and industrial manipulators.”

NVIDIA workflows join actual and artificial information

Coaching basis fashions for humanoid and different robots usually requires giant quantities of knowledge, famous NVIDIA. Teleoperation is one approach to seize human demonstration information, however it may be costly and time-consuming, it mentioned.

NVIDIA introduced a workflow that makes use of AI and Omniverse to allow builders to coach robots with smaller quantities of knowledge than beforehand required. First, builders use Apple Imaginative and prescient Professional to seize a comparatively small variety of teleoperated demonstrations.

Then, they simulate the recordings in Isaac Sim and use the MimicGen NIM to generate artificial datasets from the recordings. The builders can practice the Challenge GR00T humanoid basis mannequin with actual and artificial information, saving time and prices, mentioned NVIDIA.

Roboticists can then use Robocasa NIM within the Isaac Lab framework for robotic studying to generate experiences to retrain the robotic mannequin. All through the workflow, NVIDIA mentioned OSMO can assign computing jobs to totally different assets, eliminating weeks of administrative duties.

For example, Fourier mentioned NVIDIA’s expertise will assist it generate artificial information for coaching its general-purpose platform.

“Creating humanoid robots is extraordinarily advanced — requiring an unbelievable quantity of actual information, tediously captured from the true world,” mentioned Alex Gu, CEO of Fourier. “NVIDIA’s new simulation and generative AI developer instruments will assist bootstrap and speed up our model-development workflows.”


SITE AD for the 2024 RoboBusiness registration now open.
Register now and save.


Humanoid companies bounce on early entry

NVIDIA’s new Humanoid Robotic Developer Program presents early entry to the NIMs and OSMO, in addition to the newest releases of NVIDIA Isaac Sim on Omniverse, Isaac Lab, Jetson Thor compute, and Challenge GR00T basis fashions.

The corporate mentioned robotics builders together with 1x, Boston Dynamics, ByteDance Analysis, Discipline AI, Determine, Fourier, Galbot, LimX Dynamics, Mentee, Neura Robotics, RobotEra, and Skild AI have already joined its early-access program.

“Boston Dynamics and NVIDIA have an extended historical past of shut collaboration to push the boundaries of what’s doable in robotics,” mentioned Aaron Saunders, chief expertise officer of Boston Dynamics. “We’re actually excited to see the fruits of this work accelerating the trade at giant, and the early-access program is a unbelievable approach to entry best-in-class expertise.”

NVIDIA occasions at SIGGRAPH embody a fireplace chat between Huang and Lauren Goode, senior author at Wired, on the influence of robotics and AI in industrial digitalization. He can even have a dialog with Meta’s Mark Zuckerberg.

Latest Posts

Stay in touch

To be updated with all the latest news, offers and special announcements.