FlutterGenerator

What it does

• Built an end-to-end screenshot-to-code experiment after seeing neural networks learn visual tasks in GTA V.
• Created a training-data pipeline that collected Flutter projects, transformed widgets into runnable examples, rendered them in DartPad, and paired screenshots with Dart source code.
• Trained an image-to-sequence model using an Inception V3 encoder and LSTM decoder to generate Flutter code token by token from a UI screenshot.
• Built a Flask upload demo that ran inference and displayed the generated Dart output.
• Learned that data collection was the real challenge, and that this kind of convolutional/recurrent model was not well suited to generating convincing UI code from screenshots.

Why I built it

After watching Sentdex build neural networks that controlled GTA V from visual input, I wondered whether a model could learn to turn a UI screenshot into Flutter code.

The hard part

The hard problem was not training the network. It was creating a dataset. I built a pipeline that found Flutter projects, prepared their widgets to run in isolation, rendered them in DartPad, and paired each image with its source code.

How it worked

A pretrained image encoder processed the screenshot, while an LSTM decoder generated Dart tokens one at a time. A Flask site accepted an uploaded UI image and displayed the model’s predicted Flutter code.

What I learned

The model produced very little that was convincing. That result was useful: the project taught me that the architecture and data representation matter as much as the idea, and that standard neural networks were not a good fit for this kind of code-generation task.

How it works

1. Screenshot

   an uploaded image of a Flutter UI

2. Image encoder

   pretrained Inception V3

3. Code decoder

   an LSTM predicts Dart tokens

   one token at a time

4. Flutter code

   shown on a Flask page

Training pairs came from open-source Flutter apps rendered in DartPad.