Discover, share and engage with the world of art around you

Art Spot is a mobile app that allows users to instantly identify artwork information by taking a photo. It offers an efficient way to archive and obtain comparable works from the same artists or similar works from different artists. The app also links users to books, media, prints, and any related merchandise. Information can be shared on the go and contributes to the database through crowdsourcing each time a work is photographed. Share and track paintings. No more digging through emails, camera libraries, lost notes, or paper trails.

Challenge

We had a small team with a big idea that we had to bring to life. Our biggest challenge was collecting inventory for all artworks in New York City and deciding what features to flesh out. Originally, our target audience were high-end art collectors but after conducting further research we shifted our focus to art lovers (museum-goers, gallery, and art fair visitors) and saw an opportunity to profit from related merchandise.

Problem

• There is no efficient way for art lovers to identify artworks and take their museum/gallery/art fair experience home with them and keep their art inventory in one place.

• There is no efficient way for collectors to identify artworks and instantly obtain the information they need on the go.

Task

My task was to scope, plan, execute market research and synthesize findings to support product development in order to determine features for our application.

RESEARCH + DISCOVER

Art Market Snapshot and Competitive Landscape

The art market is a $63 billion USD yearly industry comprised of nearly 600,000 collectors, and roughly 1000 of them are responsible for half of the yearly sales ($6 billion USD). These collectors are regularly visiting galleries and art fairs to photograph and bookmark artworks of interest. Within this process, collectors share their findings with other collectors and art professionals to obtain more information on pricing and market insights that leads to acquisitions.

All current digital offerings in the competitive landscape for acquiring works were online galleries, auction houses, or marketplaces that connect sellers to buyers and receive commissions from transactions. The majority of users were made up of private dealers, advisors, galleries, and collectors from the US, Europe, and Asia.

For art lovers, we found that the only available offerings were apps that provided a database of popular artworks from museums that you can browse through and save to your digital collection.

Additional Takeaways

• Art Market approaching 63 billion USD at a 7% yearly increase.

• The top 10 museums have an annual of 57 million visitors.

• The top 25 art fairs have an annual of 3 million visitors.

• There are over 2.5 million art dealers around the globe.

• There are 600,000 mid to high-end collectors actively purchasing works on a regular basis spending between $25,000 to $15 million per artwork.

Art Collectors

After familiarizing myself with the market, I conducted research on art collectors consumer behavior. I wanted to learn:

• How often do they purchase works?

• What institutions do they primarily purchase from and in what countries?

• What type of art sales is most popular? i.e Contemporary, Masters, Ancient Artifacts, etc.

• What is their age and demographic?

The majority of art collectors are wealthy and want to stay anonymous. Obtaining such detailed information was nearly impossible. At the time I was working with an Art Advisor, and tried to get interviews from clients but many were non-compliant. It was crystal clear that they valued their privacy. We inferred some educated assumptions based on what we already knew about our clients and our experience in the industry.

Assumptions

Non-objective Information: Collectors heavily depend on dealers and advisors who hold all the cards and selectively choose their buyers.

Limited Options: Collectors have little to no information on any alternative comparable artworks on the market, impeding educated decision-making when purchasing art as a newcomer or as an investment.

High Price for the Facts: Advisory retainers and commissions are too high (often between 5 - 20%) and are not transparent or biased.

Lack of Privacy: It is difficult to obtain details on any artwork anonymously without having to interact with a dealer or advisor and giving them your personal information.  

Poor Organization: Information often comes slowly and is disorganized. There is no one-step solution to archiving, accessing, and sharing data.

Branching Out

We were able to validate that most collectors were above the age of 50 years old and were not tech-savvy. We still wanted to find a way to benefit them in the long term but we had to shift our focus to a larger audience. After seeing a staggering number of 57 million museum visitors annually from only the top 10 museums around the world, and analyzing the revenue stream from gift shop purchases, we wanted to explore and broaden our research to museum-goers and understand their motivations, behaviors, and how likely or often they purchase merchandise from gift shops.

The Museum Landscape

I researched museums’ role in the economy and community. I found that:

• 76% percent of all U.S. leisure travelers visit museums and spend 60% more on average than other leisure activities.

• 83% of museum-goers use their phone to enhance their visit by taking photos of the artworks they like.

• Museums have a strong digital presence and receive millions of online visits to their websites each year. They also serve a diverse online community, including educational institutions, teachers, parents, and students.

• 97% of Americans believe that museums are educational assets for their communities.

• 89% believe that museums contribute important economic benefits to their community.

• 96% want to maintain or increase federal funding for museums.

With these statistics, we knew that it would be feasible to work with museums because they are institutions that serve the public and value education.

Understanding User Behavior and Brainstorming

At this point, our team was set on targeting museum-goers as our primary users. I devised a research plan and a schedule for museum peak hours and seasons. I started with Van Gogh’s Starry Night at the MoMA and spent hours watching visitors crowd around the work as it is undoubtedly the most famous artwork displayed in New York City. 

Over the course of 6 months, I documented and performed contextual inquiries with guests to observe them in their physical and social environment. I occasionally would ask them questions to get a better sense of their motivations and goals.

Research Objectives

• What is the average museum-goers proficiency level in art?

• How often do they visit museums or galleries and follow the art world?

• How often are museum-goers photographing works? 

• What is the average amount of photos taken in one day for the most notable artworks? 

• How much time does the average person spend in a museum?

• What is the primary reason for visiting a museum?

• How much revenue is created through the merchandise of one artwork in a museum? i.e Books, catalogues, postcards, stationery, ticketing for specialized exhibits.

Findings

• Tourists often prioritize viewing one or a few notable works to photograph.

• Local visitors tend to only visit museums to see specific exhibitions.

• Both tourists and locals wander around museums without any specific goals to pass the time and photograph pieces that attracts them along the way and are not necessarily art enthusiasts.

• Tourists are more likely to purchase merchandise at gift shops than locals because they want to bring home a souvenir to friends and family.

• Primary attendees are below the age of 50 years old.

• Net income for retail at the Metropolitan Museum was over $2 million in 2015.
  

Painpoints

Limited Education: Visitors often read artwork descriptions but the descriptions do not provide any additional information on similar works, and artists or nearby related exhibitions.

Limited Time: A majority of tourists jam pack their schedule with activities with the time they have and often are unable to spend additional time to look through crowded gift shops to bring home a souvenir.

Cataloging Artworks: Art enthusiasts often photograph works in groups of two: artwork and the artwork description. This leads to having two separate photos in their library. They do not have an efficient way to keep track of all their photographed works in one place and have to dig through their camera library.

BRAINSTORM + DEFINE

Brainstorming Features and Solutions: How might we help?

• Provide users with a way to discover related artists and movements by connecting them with other museums and galleries nearby with similar works on view.

• Provide alerts when a work by your favorite artist is exhibited at any gallery or museum around the world.

• Provide users with the ability to save works in one place.

• Provide sharing capabilities of users virtual collection with friends and family.

• Provide users to view items and related merchandise from the museum gift shops if they want to skip the line or have little time.

Use Case:

After gathering insights, we created a simple user flow to get a clearer picture of how our product might function and what to expect from the software. We wanted to begin working with our engineers to show them how we intended the application to function.

COMPILE DATA + TEST

Testing Roadblocks: 

I began gathering images for our database to begin testing the image recognition functionality. Although a majority of images can be found online, obtaining high-resolution photographs proved to be difficult. We found that there were many factors and regulations around copyrights and that much of the inventory in museums were listed on their website without images. 

Of around 120 works tested, 1 was misidentified, and 28 were unidentified. Works that were unidentified revealed similar patterns. For example, works with segmented white space were difficult for the software to recognize, as well as color field works, and some were completely misidentified.

 Key Issues with Software

• Incompatible mediums

• Misidentification 

• Difficulty with background/foreground identification

Key Issues to Tackle:

• Angles - 3D works were difficult to capture. Some were identified from specific angles and some not at all.

• Obstructed Views - Some works were in glass displays creating a reflection making it difficult to capture.

• Lighting - Dimly lit rooms caused a slower identification for some or ‘nothing found’.

• Reflections - Works that used mirrors were difficult to capture.

• Distance and Size - Small works hung high could not be identified.

• Speedy Recognition - Some works were lagging and took a few minutes to identify.

Error Examples:

2. Mirrors/Reflexive Works

• All the paint on mirror works that we tested were unidentifiable except for one.

2. Gradient/Color Field Works

• All color field and color gradient works were unidentifiable. This was a recurring issue for all works with gradient depictions.

3. Mixed Media Monochromes

Out of the three Pinelli works tested, only one was successfully identified. This is likely due to the lack of discernible contrast/clarity with the colored works and possibly due to being at a closer distance.

4. Unconventional Frames

The three panel photographic series by John Coplans were unidentifiable. Possibly because the segmentation of the works by the white confuses the software’s identification of the image.

Left Image - UNIDENTIFIED (John Coplans, Self Portrait, Upside Down, No 4, gelatin silver print).

Middle Image - IDENTIFIED (John Coplans, Self Portrait, Back with Arms Above, gelatin silver print).

Right Image - TEMPERAMENTAL IDENTIFICATION (Marilyn Minter, Marble Study Photo, oil on canvas).

4. 3-Dimensional Sculptures

Frank Stella’s sculptures were unidentifiable, possibly due to its’ 3 - dimensionality, but it could also be due to the wall color in the preview images differing from the actual installation.

Left Image, Original Image - (Frank Stella French Tactics: An Example for All, A#2).

Right Image, Installation Shot - UNIDENTIFIED (Same work but haveifferent background from original image).

5. Misidentifications

Works with similar patterns and aesthetics were misidentified through the software.

Left Image - TESTED: (Andrew Masullo, 5533, oil on canvas).

Right Image - RESULT: (Andrew Masullo, 5811, oil on canvas).

Understanding Errors and Recommendations

After synthesizing all of our results, I had to evaluate the image recognition errors and their patterns to relay to our engineers.

1. Does the segmentation of space (in particular white) confuse the software?

2. For large scale works, would it be better to have a super hi-res image so users can just take a photo of a detail?

3. How high does the image quality need to be for the software to recognize color gradient works?

4. Does the color of the wall the artwork is mounted on affect the software’s perception of foreground and background? 

5. Will works using the same typeface (EG: Albenda works) be difficult to differentiate?

6. Could we implement a 3-D scan for sculptures?

Further Testing and Results

Out of 349 works tested, 269 were successfully identified, 67 were unsuccessful, 5 were temperamental and 8 were misidentified. Of the unsuccessful artworks, a total of 32 fell into categories outside of the algorithm’s ability. 12 were monochrome, 10 were sculptural works, 5 had a monochromatic pattern, 3 were reflective mirror works, and 2 were reflective sculptures. 

This meant that the total number of works that can fall within the identification ability of the algorithm was 317. This resulted in an 85% success rate of identifying artworks which was a huge improvement.

FINAL DELIVERABLE

REFLECTION

If I had the opportunity, I would’ve liked to conduct formal user interviews and perform usability tests with potential users. Once we established our project scope, we began gathering inventory for our database and testing the software algorithm as our stakeholders became fixated on trying to optimize the image recognition algorithm and building our artwork inventory.

My experience working on this project was invaluable. It brought to light the different challenges faced from a business perspective and from a technical standpoint. An idea of this magnitude in its’ early stages needed a thorough business plan with more funding, patience and time. It taught me the importance of strategizing your product roadmap and risk management.