Open Datasets
Mobbin Dataset
AI tools for designers
Open Datasets
Mobbin Dataset
AI tools for designers
TAGS
A design system is a co-visual language for designers and engineers, and these days, it has become much common that it can be easily created and used by anyone. However, the design system still has major hurdles. The point is that the more components and patterns are included in the design system, the more complex it becomes. This is where the project has begun. To create an assistant in which AI learns numerous design elements over and over instead of a human, and automatically recommends the rules of the design system on the designer’s screen like a spell checker.
About
Datumo provides high quality data for smarter AI. As part of Datumo's Data Sponsorship Program, Datumo cooperated with a study group of Naver and Line designers within Naver in building the following dataset.
The team started out as a study group of Naver and Line designers within Naver. The group consists of professionals in UX design and Design system(Library), and have been focusing on various ‘1qwH’ technology that could solve the problems designers have been facing during the process of developing LINE and other global designs and products. As part of the study, the group made an attempt of applying object detection to designing, and distinct potentials were observed via prototyping. Confident with the possibility of improving the design industry with appropriate open source datasets, the team applied for Datumo’s “2021 Data Sponsorship Program”. The team is continuing their studies on applying machine learning to designing, and aims to publish interesting results soon.
“The most difficult parts in object detection project was data collecting and labeling. AI performance depends more on quality than quantity of data. In that sense, we were extremely fortunate to meet Datumo through the data sponsorship program. Datumo did not simply provide a crowd-sourcing platform. They started out by passionately discussing with our researchers to systematically build user guidelines. We knew from our past experiences of uploading PDF guidelines on other platforms, that even coming up with the outline for the guidelines were difficult for inexperienced researchers. However, we were able to write quality guidelines using Datumo’s project managers’ tips, which led to efficient labeling process”
• Due to copyright issues, only sample data will be shared.
• Publishing of the full dataset is to be announced.
• Data labeling of total 11,999 images
• 50,000+ bbox
• Label bounding boxes according to thirteen UI categories from iOS mobile application screenshots
• Write user guidelines on each UI category
• Extract common characteristics of UI from different mobile applications, via PoC
ex) Most floating buttons have shadows, are round-shaped, and are located on the bottom of the screen
• Separate the categories with more edge cases from those with less, for higher accuracy
• Final inspection via LabelMe
• Post-processing of data using codes
Data Collection
• Project started from a study group of Naver and Line Plus designers
• Aim to develop an AI tool for designers
• Aim to build an AI model trained with bounding boxes drawn around the UI elements on iOS
• Datumo was responsible for drawing bounding boxes around thirteen UI elements in provided raw data(mobile iOS screenshots)
Data were collected and labeled using Cash Mission, Datumo's crowd-sourcing platform.
{
"version": "4.5.6",
"flags": {},
"shapes": [
{
"label": "page control",
"points": [
[
646.8384074941451,
1210.3044496487119
],
[
794.6135831381732,
1230.679156908665
]
],
"group_id": null,
"shape_type": "rectangle",
"flags": {}
},
{
"label": "text field",
"points": [
[
57.732394366197184,
1556.3380281690143
],
[
1383.0845070422536,
1674.6478873239437
]
],
"group_id": null,
"shape_type": "rectangle",
"flags": {}
},
{
"label": "radio",
"points": [
[
1286.4197530864196,
1735.8024691358023
],
[
1370.9677419354837,
1823.6559139784947
]
],
"group_id": null,
"shape_type": "rectangle",
"flags": {}
},
{
"label": "radio",
"points": [
[
1285.8024691358023,
1961.111111111111
],
[
1370.6403823178014,
2045.3118279569899
]
],
cs
UI1.json
<!--?xml version="1.0" ?-->
UI1.jpg
./json/UI1.json
roboflow.ai
1440
2560
3
0
<object width="300" height="150"> <name>page control</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>646.8384074941451</xmin> <ymin>1210.3044496487119</ymin> <xmax>794.6135831381732</xmax> <ymax>1230.679156908665</ymax> </bndbox> </object> <object width="300" height="150"> <name>text field</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>57.732394366197184</xmin> <ymin>1556.3380281690143</ymin> <xmax>1383.0845070422536</xmax> <ymax>1674.6478873239437</ymax> </bndbox> </object> <object width="300" height="150"> <name>radio</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>1286.4197530864196</xmin> <ymin>1735.8024691358023</ymin> <xmax>1370.9677419354837</xmax> <ymax>1823.6559139784947</ymax> </bndbox> </object> <object width="300" height="150"> <name>radio</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>1285.8024691358023</xmin> <ymin>1961.111111111111</ymin> <xmax>1370.6403823178014</xmax> <ymax>2045.3118279569899</ymax> </bndbox> </object> <object width="300" height="150"> <name>radio</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>1286.4197530864196</xmin> <ymin>2187.037037037037</ymin> <xmax>1370.3703703703702</xmax> <ymax>2269.135802469136</ymax> </bndbox> </object> <object width="300" height="150"> <name>radio</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>1285.8024691358023</xmin> <ymin>2409.876543209876</ymin> <xmax>1369.2688172043006</xmax> <ymax>2494.0000000000005</ymax> </bndbox> </object>UI1.xml
{
"version": "4.5.6",
"flags": {},
"shapes": [
{
"label": "text field",
"points": [
[
77.77777777777777,
160.49382716049382
],
[
1361.7283950617284,
354.32098765432096
]
],
"group_id": null,
"shape_type": "rectangle",
"flags": {}
},
{
"label": "text field",
"points": [
[
79.01234567901234,
390.12345679012344
],
[
1363.4408602150536,
586.0215053763441
]
],
"group_id": null,
"shape_type": "rectangle",
"flags": {}
},
{
"label": "text field",
"points": [
[
75.83333333333334,
620.8333333333334
],
[
1361.0294117647059,
818.3823529411765
]
],
"group_id": null,
"shape_type": "rectangle",
"flags": {}
},
{
"label": "text field",
"points": [
[
79.01234567901234,
854.3209876543209
],
[
1364.1975308641975,
1050.6172839506173
]
],
"group_id": null,
"shape_type": "rectangle",
"flags": {}
},
UI2.json
<!--?xml version="1.0" ?-->
<annotation>
<filename>UI2.jpg</filename>
<path>./json/UI2.json</path>
<source>
<database>roboflow.ai</database>
<size>
<width>1440</width>
<height>2560</height>
<depth>3</depth>
</size>
<segmented>0</segmented>
<object>
<name>text field</name>
<pose>Unspecified</pose>
<difficult>0</difficult>
<occluded>0</occluded>
<bndbox>
<xmin>77.77777777777777</xmin>
<ymin>160.49382716049382</ymin>
<xmax>1361.7283950617284</xmax>
<ymax>354.32098765432096</ymax>
</bndbox>
</object>
<object>
<name>text field</name>
<pose>Unspecified</pose>
<difficult>0</difficult>
<occluded>0</occluded>
<bndbox>
<xmin>79.01234567901234</xmin>
<ymin>390.12345679012344</ymin>
<xmax>1363.4408602150536</xmax>
<ymax>586.0215053763441</ymax>
</bndbox>
</object>
<object>
<name>text field</name>
<pose>Unspecified</pose>
<difficult>0</difficult>
<occluded>0</occluded>
<bndbox>
<xmin>75.83333333333334</xmin>
<ymin>620.8333333333334</ymin>
<xmax>1361.0294117647059</xmax>
<ymax>818.3823529411765</ymax>
</bndbox>
</object>
<object>
<name>text field</name>
<pose>Unspecified</pose>
<difficult>0</difficult>
<occluded>0</occluded>
<bndbox>
<xmin>79.01234567901234</xmin>
<ymin>854.3209876543209</ymin>
<xmax>1364.1975308641975</xmax>
<ymax>1050.6172839506173</ymax>
</bndbox>
</object>
<object>
<name>button</name>
<pose>Unspecified</pose>
<difficult>0</difficult>
<occluded>0</occluded>
<bndbox>
<xmin>77.77777777777777</xmin>
<ymin>1118.5185185185185</ymin>
<xmax>1361.7283950617284</xmax>
<ymax>1298.7654320987654</ymax>
</bndbox>
</object>
</annotation>UI2.xml
{
"version": "4.5.6",
"flags": {},
"shapes": [
{
"label": "icon",
"points": [
[
54.782608695652165,
269.5652173913043
],
[
139.1304347826087,
353.9130434782608
]
],
"group_id": null,
"shape_type": "rectangle",
"flags": {}
},
{
"label": "page control",
"points": [
[
655.6291390728477,
366.2251655629139
],
[
785.4304635761589,
402.64900662251654
]
],
"group_id": null,
"shape_type": "rectangle",
"flags": {}
},
{
"label": "icon",
"points": [
[
1151.3966480446927,
1484.3575418994412
],
[
1240.782122905028,
1573.1843575418993
]
],
"group_id": null,
"shape_type": "rectangle",
"flags": {}
},
{
"label": "icon",
"points": [
[
1281.5642458100558,
1484.9162011173185
],
[
1367.0391061452515,
1567.5977653631285
]
],
"group_id": null,
"shape_type": "rectangle",
"flags": {}
},
UI3.json
<!--?xml version="1.0" ?-->
UI3.jpg
./json/UI3.json
roboflow.ai
1440
2560
3
0
<object width="300" height="150"> <name>icon</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>54.782608695652165</xmin> <ymin>269.5652173913043</ymin> <xmax>139.1304347826087</xmax> <ymax>353.9130434782608</ymax> </bndbox> </object> <object width="300" height="150"> <name>page control</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>655.6291390728477</xmin> <ymin>366.2251655629139</ymin> <xmax>785.4304635761589</xmax> <ymax>402.64900662251654</ymax> </bndbox> </object> <object width="300" height="150"> <name>icon</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>1151.3966480446927</xmin> <ymin>1484.3575418994412</ymin> <xmax>1240.782122905028</xmax> <ymax>1573.1843575418993</ymax> </bndbox> </object> <object width="300" height="150"> <name>icon</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>1281.5642458100558</xmin> <ymin>1484.9162011173185</ymin> <xmax>1367.0391061452515</xmax> <ymax>1567.5977653631285</ymax> </bndbox> </object> <object width="300" height="150"> <name>slider</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>59.11475409836066</xmin> <ymin>1709.8360655737706</ymin> <xmax>1382.4324324324325</xmax> <ymax>1770.2702702702702</ymax> </bndbox> </object> <object width="300" height="150"> <name>button</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>636.8159203980099</xmin> <ymin>2086.567164179104</ymin> <xmax>802.4875621890546</xmax> <ymax>2252.238805970149</ymax> </bndbox> </object> <object width="300" height="150"> <name>icon</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>967.6616915422884</xmin> <ymin>2132.835820895522</ymin> <xmax>1074.1293532338307</xmax> <ymax>2201.9900497512435</ymax> </bndbox> </object> <object width="300" height="150"> <name>icon</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>367.6616915422885</xmin> <ymin>2134.328358208955</ymin> <xmax>476.61691542288554</xmax> <ymax>2201.9900497512435</ymax> </bndbox> </object> <object width="300" height="150"> <name>button</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>1252.73631840796</xmin> <ymin>2116.9154228855723</ymin> <xmax>1346.7661691542287</xmax> <ymax>2215.4228855721394</ymax> </bndbox> </object> <object width="300" height="150"> <name>button</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>93.5323383084577</xmin> <ymin>2115.92039800995</ymin> <xmax>187.56218905472636</xmax> <ymax>2214.9253731343283</ymax> </bndbox> </object> <object width="300" height="150"> <name>icon</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>91.54228855721392</xmin> <ymin>2312.9353233830843</ymin> <xmax>190.04975124378106</xmax> <ymax>2415.92039800995</ymax> </bndbox> </object> <object width="300" height="150"> <name>icon</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>667.6616915422885</xmin> <ymin>2310.4477611940297</ymin> <xmax>773.6318407960198</xmax> <ymax>2417.9104477611936</ymax> </bndbox> </object> <object width="300" height="150"> <name>icon</name> <pose>Unspecified</pose> <difficult>0</difficult> <occluded>0</occluded> <bndbox> <xmin>1251.2437810945273</xmin> <ymin>2311.9402985074626</ymin> <xmax>1350.7462686567162</xmax> <ymax>2413.930348258706</ymax> </bndbox> </object>UI3.xml
AI Tools for designers
CC BY-SA
Reusers are allowed to distribute, remix, adapt, and build upon the material in any medium or format, even commercially, so long as attribution is given to the creator. If you remix, adapt, or build upon the material, you must license the modified material under identical terms.
https://creativecommons.org/licenses/by-sa/3.0/deed.en