Creación de una arquitectura de redes neuronales con un enfoque modular para la generación de código fuente HTML y CSS a partir de Mockups

Abstract

This work aims to develop a neural network architecture capable of translating a graphical interface screenshot into HTML and CSS code, in order to automate the mentioned activity, since the manual elaboration of this one brings certain disadvantages (Chapter 1 and 2). Although (Beltramelli, 2017), (Wallner, 2018) and (silverstar94, 2019) have proposed neural networks as a proof of concept for the same purpose, this project proposes an architecture with a modular approach and the ability to generate a greater number of tags compared to previous work (Chapter 6). To carry out the training and evaluation of the networks, artificial data and web pages extracted from the Internet were used (Chapter 8 and 9). Later, in the case of HTML, Pix2Code, YOLOv3 and RetinaNet architectures were selected for the prediction of elements (Chapter 10). On the other hand, CSS properties were grouped according to the similarities of their effects, defining grammars, building datasets based on them and training for the prediction of the selected styles (Chapter 11). As a main result, it was found that the best architecture for the detection of HTML elements is YOLOv3. In the case of CSS, architectures with the VGG16 encoder and trained with a batch size of 16 obtained the best performance. Finally, the best architectures obtained were integrated using Rust to perform the generation of HTML source code and CSS properties, receiving as input an image of 1280x1280 pixels (Chapter 12).