🔍 Read the full analysis: Can Fine-Tuning Improve 3D Generation Models? on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Hugging Face has published scripts and container images for fine-tuning its TRELLIS.2 3D generation model on custom asset datasets, locally or through Hugging Face Jobs. An example uses LoRA adapters and 188 footwear objects to target shoe geometry, including soles, but the source does not report whether the experiment improved results.
As described in the original fine-tuning guide, Hugging Face has published scripts and containers for fine-tuning its TRELLIS.2 3D generation model on custom asset collections, with an example that targets shoe geometry using 188 footwear objects. The release gives developers a local or cloud-based way to try category-specific adaptation, but the accompanying material does not report a completed evaluation showing whether the fine-tuning improved generated shoes.
The example uses low-rank adaptation (LoRA), a method that adds trainable adapter parameters while keeping the original model weights frozen. Hugging Face says this reduces the number of parameters being trained and the memory required for gradients and optimizer state; the scripts use the PEFT library. The dataset is a subset of 188 footwear objects from Google Scanned Objects, selected to test whether adaptation could address a specific issue seen in initial tests: shoes looked plausible overall, but their soles were inaccurate.
The training plan modifies TRELLIS.2’s sparse-structure and shape stages, which influence an object’s form and detailed geometry. It leaves the texture-generation stage unchanged. The source says the footwear assets have diffuse color maps but lack the separate material maps needed to train that stage, and that the base model already produced satisfactory colors and textures for the task. This explains the authors’ chosen setup; it is not evidence that this component selection leads to better results.
Users can run preprocessing and training with Hugging Face Jobs or use Docker commands on a local machine with a suitable GPU. The described workflow requires uploading a mesh dataset to the Hugging Face Hub, a Hugging Face token, and account access to gated DINOv3, which the pipeline uses to encode input images. The supplied account outlines these steps but does not provide hardware requirements, costs, or a final result from the shoe experiment.
Fine-tuning TRELLIS.2 for custom 3D assets
Hugging Face released scripts and containers to adapt TRELLIS.2 to specific object categories. The footwear example outlines an experiment, but reports no completed results.
What the workflow targets
The example addresses shoes that look plausible but have inaccurate soles.
Coarse shape
LoRA adapters target sparse structure to adapt the model to footwear.
Detailed shape
The shape stage generates geometry; the example aims to improve shoe details such as soles.
Stage left unchanged
The dataset lacks separate material maps, and the authors say base colors and textures are satisfactory.
How TRELLIS.2 builds a mesh
The described pipeline turns an input image into a textured mesh.
BiRefNet removes the background, then DINOv3-L encodes the image.
Sparse structure guides the model’s detailed geometry generation.
A separate texture stage generates materials before the outputs are combined into a GLB mesh.
Ways to run the experiment
Teams with suitable assets can choose a local or cloud workflow.
- Use the provided Docker commands with your own GPU.
- Run preprocessing and training scripts on your machine.
- Run containerized workloads on cloud hardware.
- Upload the mesh dataset to the Hugging Face Hub.
What teams need
Access and evaluation matter alongside the training setup.
- A Hugging Face token and access to gated DINOv3.
- A comparison of pretrained and fine-tuned outputs, including held-out footwear.
- Clear geometry criteria and reported training settings, hardware, and costs.
Testing Category-Specific 3D Geometry
The release addresses a practical problem in 3D generation: a model can create an object that looks broadly right while still getting task-relevant geometry wrong. For footwear, a poorly shaped sole may matter more than a convincing color or surface texture. The example tests whether developers can use their own object collections to focus a general-purpose system on such a narrower requirement.
That distinction could matter for teams working in robotics, simulation, and media, where generated assets may need consistent shapes rather than visual plausibility alone. Still, the available source describes an experiment, not evidence that fine-tuning solves the problem. Without reported measurements or examples, readers cannot judge whether the resulting soles improved, whether any gains came at the cost of other features, or how much preparation and compute the workflow demands.
The scripts and containers may make it easier to try the approach across local and cloud hardware. Their practical value will depend on whether others can reproduce the process, how the model performs on footwear outside the training subset, and whether the improvement—if there is one—meets a real production need.
Products Worth Considering
Logitech MX Mechanical Wireless Illuminated Keyboard, Clicky Switches
Jackery Explorer 300 Portable Power Station with 40W Solar Panel, 292Wh
DJI Osmo Pocket 3 Vlogging Camera with 1-Inch CMOS Sensor and 4K/120fps Video
As an affiliate, we earn on qualifying purchases.
How TRELLIS.2 Builds 3D Assets
TRELLIS.2 is presented in the source as a successor to TRELLIS, a system built around a shared latent representation intended to support multiple 3D output formats through decoders. The stated aim is to avoid needing a separate generation model for each representation. TRELLIS.2’s described pipeline produces a textured mesh that can be exported as a GLB file.
In the workflow outlined by the post, BiRefNet removes the image background and DINOv3-L encodes the image. A sparse-structure flow model predicts a coarse form, which guides a shape flow model that generates detailed geometry. A separate texture flow model generates surface materials. The shoe example adds LoRA adapters to the structure and shape diffusion transformers, while leaving texture generation untouched.
The choice reflects the specific target: the experiment is about shoe shape and soles, not surface appearance. The source does not establish that this is the best setup, compare it with other training configurations, or report whether the selected stages changed the output as intended.
“The example targets footwear geometry, including inaccuracies in generated soles.”
— Hugging Face, in the published fine-tuning example
No Shoe Evaluation Yet Reported
The supplied material does not say whether training was completed or whether the adapted model produced more accurate soles. It includes no before-and-after examples, benchmark, or independent assessment, and does not specify the evaluation set or criteria used to judge geometry. The reported sole issue comes from the authors’ initial tests, whose details are not given.
Other practical questions also remain open: the source does not state the hardware, training time, compute cost, or data-preparation effort required. It does not report how well an adapted model would generalize to shoes outside the 188-object subset, or whether changes to structure and shape affect other output qualities. As a result, the announcement confirms the availability of a workflow, not a demonstrated improvement in 3D generation.
Running and Evaluating the Workflow
Developers with suitable mesh assets can adapt the published scripts, prepare a dataset on the Hugging Face Hub, and run preprocessing and training either locally or through Hugging Face Jobs. They will need a Hugging Face token and access to gated DINOv3. The source also describes a walkthrough, but the supplied information does not give a schedule for further results.
The next evidence needed is a reported evaluation: whether fine-tuning changed sole geometry, how the output compares with the pretrained model, and whether any gains hold for footwear not included in training. Details on compute, reproducibility, and the evaluation method would help developers decide whether the workflow is practical for their own 3D asset tasks.
Key Questions
What did Hugging Face release?
It published scripts and containers for fine-tuning TRELLIS.2 on custom 3D asset datasets, with options to run training locally or through Hugging Face Jobs.
Did fine-tuning improve the generated shoes?
The supplied source does not report a completed outcome. It describes the experiment but gives no benchmark, before-and-after comparison, or measured improvement.
What does the footwear example train?
It uses LoRA adapters on TRELLIS.2’s sparse-structure and shape stages, using a subset of 188 footwear objects from Google Scanned Objects. The texture stage is left unchanged.
What do developers need to run the workflow?
They need a mesh dataset uploaded to the Hugging Face Hub, a Hugging Face token, and access to gated DINOv3. Local runs also require a suitable GPU; cloud runs are described through Hugging Face Jobs.
What remains unknown about the experiment?
The source does not report training time, hardware, cost, evaluation criteria, or generalization to footwear outside the training subset. It also does not show whether the generated soles became more accurate.
Primary source: Hugging Face · via ThorstenMeyerAI.com
