Plant Hunter - A Prototype

I've been working on something fun over the last few weeks. As someone who likes to build stuff in the real world as well as on the internet I've often lamented that there wasn't a larger intersection between the gardening world and the software engineering world. As I've admitted in the past I've got a bit of  'collector personality disorder' happening when it comes to buying plants... I'm pretty sure some of you do as well :) A big part of collecting things you love is sharing your "finds" with kindred souls. Even if you don't buy it - there is still value in spotting a plant and sharing the details with others. With that context I introduce ... Plant Hunter which is a way to document the price and location of those treasures and share them with other plantaholics.

 



Here I am at the store that shall remain nameless taking a picture and capturing details in the field.


For this prototype I built just enough interface to understand how to best structure the workflow so apologies if the screens below look basic :) Ok - lets get started!

 

Step 1: Take a picture through the web app. (It's also possible to upload existing pictures already on your phone.)

Step  2: Crop away all but the pertinent details found in the plant label. This is important because it helps with accuracy during the image to text translation step. ( don't worry the original image will be saved) To help images tilted off level I've added some simple controls that rotate the source by a few degrees at a time.

Step 3: Once the image is cropped some behind the scenes pre-processing automatically takes place to help with image to text translation quality.

The next automatic step is OCR (Optical Character Recognition) which is the magic sauce used to tease out words from the image.

We may get words from the image but it's difficult to make assumptions about a plant label's structure because they come in so many varieties! There is no universal plant label format. They may or may not contain information about pot size, price, grower, retail nursery, a bar code, a QR code, water and sun requirements, use in the garden, etc, etc. Of course how and where this information is organized will vary from label to label as well. One assumption I do make is that the label contains the plant name.

 


The quality of text so far has been highly variable in part because the labels are sticking to the curved surface of a pot which introduces "skew". Curved words make it hard for the OCR engine to do it's job. This is something I need to work on but for now taking a decent, close up shot helps a lot.


When there isn't enough information to determine the plant name the user can always fine tune the selection from a list of plants that contain the same keywords.


Step 4: collects information about where you spotted the plant.

To determine the location I used a combination of the phone's current GeoLocation (i.e. latitude and longitude coordinates) and the 'Places' service from Google which provides information about stores and other places of interest. To make a long story short - we provide the user's coordinates and the store name hint to a specialized Google service that's expert at everything to do with places on a map. The Google service hands back what it finds which may be nothing, one result or multiple results. 
 

Above is the last step - a quick confirmation before your "field report" (including pictures) is saved to a database in the cloud.

 Once saved, the plant details will be aggregated with other field reports and made available to fellow plantaholics for searching, sorting, viewing on maps etc ... that's the next step whose details are not fully flushed out just yet. 

For good measure - below is a screenshot showing how the plant and it's details look in the cloud store (database).


 

Putting my user hat on - I think it would be interesting to discover availability and price of any given plant across an ever growing set of retail outlets. It's one thing to read about a plant and plan on using it in your garden someday - it's another thing to know where you can buy the plant, if it's competitively priced, and possibly even historic price information. Frankly just knowing a little bit about what my local nursery has in stock would be useful. Currently the only alternative is to visit the store.

To be honest this prototype is not ready for prime time but I wanted to share the general idea and solicit some feedback. What do you think? Is this something you would even use? Would you want to save other details about a plant? Do you think a picture of the plant itself is worth saving? Do you think taking lots of pictures upfront on your phone and saving the rest of the workflow for a desktop experience would be better? 

Looking forward to any feedback!




Comments

  1. Cool idea. I'm also one foot in the tech world, one in the gardening world. I've always been a record keeper, so I tend to snap photos of plants and labels, so I can later go back and ID the photos/remember what I've planted. And that backlog of reviewing and ID-ing my photos can get really long. Like, years long.
    One thing I've found in my UX day job is that switching from a phone to a computer is an extremely leaky funnel and people drop out a lot if they can't do one quick interaction and be done with it. That, and people are much more motivated if there's something in it for them. For instance, if they have a profile/feed/social feedback.
    At any rate, I think this is a really interesting idea. I could talk plant/data nerd stuff all day. Good luck with the project. I'll be interested to see where it goes.

    ReplyDelete
    Replies
    1. Thanks Megan for the insights! I'm happy to know there are other plant/tech nerds in the world. The 'leaky funnel' observation is a good one - most likely only the most dedicated of users would make it from the mobile to desktop experience - possibly because they might want to document large numbers of plants in a single session. To your 'quick interaction' point - yeah that's a problem currently and I've tried to minimize what the user needs to do in order to complete a workflow. For example helping them look up the store/place name, trying to be smart about how keywords are identified in text and matched to a known list of plants and automatically kicking off some processes. It's not enough tho... particularly the cropping step can be painful! I've thought about using a machine learning service to auto-magically determine where the label is on a plant pic.

      Lastly the 'something in it for the them' observation - yes I agree with that too! A PM I worked alongside once mentioned successful apps should appeal to one of the seven deadly sins e.g. 'vanity' etc. I think there's a lot that could happen around users "owning" a place or vertical of plants, or individual plant based on quantity and frequency of data they submit.

      Delete
  2. A tool for urban plant hunters is a cool idea, Hans! It might even encourage outlets, especially the big box stores, to stock more unusual specimens if it began to drive customer traffic. Of course, that depends heavily on the number of users in a given region. A search feature that allowed users to quickly and easily scan stock in their own areas would be important and I like the idea of including a plant photo as sometimes one doesn't know one wants something until seeing it ;) Personally, I'd probably prefer to handle the workflow on my desktop rather than fiddle with tweaking something on my tiny and ancient cell phone.

    ReplyDelete
    Replies
    1. "A search feature that allowed users to quickly and easily scan stock in their own areas would be important ..." I feel the same way. Currently I have a informal list of plants I'd like to buy and other than ordering online my technique for finding those plants is to go to a store / plant sale etc and hope it's there.

      Your preference for a desktop is a good reminder that not everyone has a powerful computing device in their pocket ( a new'ish phone) - or even young eyes to manipulate a small interface! I don't have young eyes anymore :( In this case the workflow could be shortened to just taking a pic and saving it to the cloud store (along with lat/lng info) Later a user could log into a website on their desktop and finish workflow.

      Delete
  3. hmmm...tough question, and tough problem to solve. Offhand, for an extremely lazy person such as myself, seems like a lot of work for...???

    Devils advocate: Why would a person not just google "aloe glauca for sale near me"

    Have a look at a regional (PNW) site called plantlust, (there are others that do basically the same thing) which is driven not by shoppers but by suppliers--its a groups of nurseries that list plants, and the website will tell you about the plant, who has it, and link to where you can buy it from them. Maybe there's a place for a Bay Area site that does the same?

    I'm not on FB, but don't people use that for plant groups and selling plants and so forth? I have no idea how FB works, but can a user say "does anyone have aloe glauca for sale" and get responses for that?

    a tough nut to crack. best wishes!

    ReplyDelete
    Replies
    1. Unfortunately local retail nurseries don't keep an online, running inventory of plants and prices (at least as far as I've seen). Some do post info about 'new arrivals'. Generally if you want to see what's for sale at local retail outlets it involves a car trip. Some larger wholesale / retail operations ( for example Waterwise Botanicals in SD county) will send out a monthly pdf of stock but that is not something that will show up in a search engine. Online only operations like MountainCrest Gardens will show up but they are not local. So a search like "[plant X] for sale near me" won't yield good results for most people.

      I've visited PlantLust a few times and it seems very well done but the difference is as you mentioned - "suppliers" (remote and through an online store) vs local retailers. Same with a FB group. It's fun to order online but I also like to visit a nursery in person where I can pick the plant I want, enjoy the atmosphere and perhaps discover new things.

      So basically the value add here is some insight into what's available locally. To make that happen the mom and pop places, big box stores etc would need to diligently list what they have available online... which hasn't happened. My thought was that alternatively an app could enable a relatively large # of people to put forth a little effort and achieve the same end. There's possibly other incentives as well that align with why we spent time writing blog posts: community, sharing useful information and pride in sharing what we've created (or found!).

      All that said - I don't really know if anyone (other than me) would use this idea. I've always wanted something online to tell me what's in stock at the ~20 or so local places I can visit (and who has the best prices!)

      Delete

Post a Comment

Popular Posts