Harnessing AI For Watercolour Art: A Guide With TRL And OpenEnv

🔍 Read the full analysis: Harnessing AI For Watercolour Art: A Guide With TRL And OpenEnv on ThorstenMeyerAI.com

An independent engineer has released a complete, open-source reproduction of Surya Narreddi’s viral watercolour art AI, built using TRL and OpenEnv on Hugging Face. This release includes all datasets, training scripts, and models, making the process transparent and accessible for further experimentation in aesthetic reinforcement learning.

The reproduction replicates Narreddi’s model, which gained widespread attention in August after a viral video showcasing AI-generated watercolour paintings. For more details, see the original analysis on Thorsten Meyer AI’s coverage. The project trains a Qwen language model with reinforcement learning against a composite aesthetic reward, combining multiple scoring components including style similarity and human preference models. All artifacts—datasets, environment scripts, and trained models—are openly available on Hugging Face, enabling the community to verify, adapt, and extend the work.

The core technical approach involves training the model to generate JavaScript code that produces watercolour-style paintings using the p5.brush library. The reinforcement learning process rewards the model based on criteria such as code correctness, length, stylistic similarity to reference images, and human-like aesthetic preferences. The reward mix includes a gate to prevent cheating, a length penalty, a style judge based on a vision-language model, and a preference model trained on human choices. The training runs on Hugging Face’s infrastructure, using a pipeline that executes 110 steps with multiple episodes per step. This process is similar to techniques discussed in the original analysis.

This open release addresses a key barrier in the original project: the lack of publicly available artifacts. By providing the datasets, reference pools, and training scripts, the reproduction enables others to analyze, modify, and improve upon the original work, fostering open research into aesthetic reinforcement learning and AI art generation.

Implications for AI Art and Reinforcement Learning

This development marks a significant step in democratizing AI art creation, allowing researchers and artists to explore reinforcement learning beyond traditional, verifiable tasks. The project demonstrates that models can be optimized against subjective aesthetic preferences, opening new avenues for AI-generated art that emphasizes style and taste rather than correctness.

Furthermore, the open release provides transparency and reproducibility, addressing common barriers in AI art research. It enables detailed inspection of the code, datasets, and reward mechanisms, which can lead to improved techniques and broader experimentation. The approach also exemplifies how language models can be trained to produce human-readable, editable code that creates visual art, offering a unique intersection of programming, art, and machine learning.

Overall, this work highlights the potential for reinforcement learning to align AI outputs with subjective human preferences, challenging the dominance of ‘perfect’ images generated by mainstream models and emphasizing handcrafted, imperfect, and expressive artworks.

Background on AI Art and Reinforcement Learning

The project builds on a long history of generative AI art, tracing back to early works like DeepDream (2015) and neural network portraits by Mario Klingemann. These early experiments explored the medium’s creative potential and often involved curated datasets or handcrafted tools.

In 2023, Surya Narreddi’s viral watercolour video brought renewed attention to AI art, showcasing a model that writes JavaScript code to produce watercolour effects with natural brushstrokes. The original project used a language model trained with reinforcement learning against aesthetic criteria, but lacked open artifacts for community verification.

This reproduction extends the initial work by making all components openly accessible, thus providing a resource for further exploration of aesthetic reinforcement learning and AI-generated art’s potential to mimic human taste and style.

“This open release aims to foster transparency and collaboration in AI art, enabling others to verify and build upon the original watercolour model.”

— Thorsten Meyer, project author

Unresolved Questions About Model Performance

It remains unclear how the reproduced model’s outputs compare quantitatively to Narreddi’s original artworks in terms of style fidelity and aesthetic quality. The project’s authors have not provided a detailed evaluation or final comparison metrics, and the effectiveness of different reward mixes is still under analysis.

Additionally, the full technical report from Narreddi, which could clarify the original design choices and technical nuances, has not yet been published. The extent to which the open reproduction captures the full artistic intent of the viral video remains to be seen.

Next Steps for Community Validation and Development

The next phase involves community engagement: researchers and artists are encouraged to test, modify, and extend the open artifacts. The release serves as a baseline for exploring how reinforcement learning can be tuned for subjective aesthetic goals.

Further updates are expected as Narreddi publishes a comprehensive technical report, which will likely include detailed evaluations and insights into the model’s performance. Additional experiments comparing different reward configurations and artistic styles are also anticipated.

Ultimately, the project aims to foster a broader understanding of how AI can be trained to produce art that aligns with human taste, moving beyond pixel-perfect images to more expressive and imperfect works that resonate on a subjective level.

Key Questions

What is the main achievement of this open reproduction?

The project provides a fully open, end-to-end implementation of Narreddi’s viral watercolour AI, including datasets, training scripts, models, and environment setups, enabling transparent research and development.

How does reinforcement learning differ in this project compared to traditional uses?

Instead of optimizing for verifiable correctness, the model is trained to align with subjective aesthetic preferences, using a reward system based on style similarity and human preferences.

What tools and frameworks are used in this reproduction?

The project employs TRL (Training Reinforcement Learning) and OpenEnv for environment management, with models trained on Hugging Face infrastructure, including Qwen models and vision-language scoring systems.

Will this work be applicable to other forms of AI art?

Yes, the open approach and methodology can be adapted to different artistic styles, media, and aesthetic criteria, fostering broader experimentation in AI-generated art.

When can we expect the full technical report from Narreddi?

The report has been promised but has not yet been published. It is anticipated in the near future and will likely provide detailed evaluations and insights into the original model’s design and performance.

Primary source: Hugging Face · via ThorstenMeyerAI.com