They basically took the same idea and instead of just producing a hack finished a nice full version of the idea - very nice write up!
But, they don't have any web UI for it yet, so I am getting tempted to revive my dormant side project and likewise make a proper polished version of it with a web interface. If anyone on here is a talented web dev with an interest on working on this as a side project (for free, and for fun, though we could explore monetization if it works), feel free to get in touch! (ps extra get in touch if you are near south bay area/Stanford)
PS Incidentally, I independently came up with the same idea as in Deep Interactive Object Segmentation (https://arxiv.org/abs/1603.04042) and implemented it for my Stanford CS 229 (Machine Learning) project first - the hack came later. The ObjectCropBot hack allows only cropping and not clicking because it was faster to hack in that, but I think it is ideal to allow both cropping to constrict around target object and clicking (clicking alone leaves too much ambiguity , eg do you want to crop the person, or their shirt, or a subset of that shirt).
Pizza Photo Editor is still early-stage -- we're still adding features and increasing the coverage of different object classes in our training data set -- but the core tech is there, and it does have an interactive web UI, so it's already fun to play with. :)
Looking at the example inputs & outputs for ObjectCropBot, it's evident that the challenge is not just detecting the object and clipping it approximately, but tracing a tight precise boundary around it. We've found that DNN/ConvNet-based approaches don't offer the necessary precision, so it's necessary to perform some pre- or post-processing using other Computer Vision techniques. At Object AI, we've developed an "object boundary deduction" pipeline that combines ConvNets with other tech. It's interesting that the blog post that you've linked to makes the same observation!
Wow, this is great! Maybe i'll shelf my efforts a bit , if your online photo editor is this good... I did Google around and find some of these GraphCut-esque refinement techniques, but have yet to find an easily usable one that gave you a good crop with just an outline or tap (or both) - kudos for making one!
As to the lack of precision of ObjectCropBot, I think a lot of that is due to it running based on DeepMask and not SharpMask; Facebook's insight to run the final low-res features back through increasingly less downsized images with sort of skip connections is a good one, and I'd bet the resolution of the results would be a lot better if I just hacked it to use that instead of DeepMask (I think the thing I linked just uses that, pretty much).
Reading the title I was interested to see how this thing filled in the background after cropping an object from a photo... Alas, it's detecting an object in the selected region and providing a cut-out of that object to use elsewhere.
Still impressive, just needs a better name. There are two problems in Computer Science...
They basically took the same idea and instead of just producing a hack finished a nice full version of the idea - very nice write up!
But, they don't have any web UI for it yet, so I am getting tempted to revive my dormant side project and likewise make a proper polished version of it with a web interface. If anyone on here is a talented web dev with an interest on working on this as a side project (for free, and for fun, though we could explore monetization if it works), feel free to get in touch! (ps extra get in touch if you are near south bay area/Stanford)
PS Incidentally, I independently came up with the same idea as in Deep Interactive Object Segmentation (https://arxiv.org/abs/1603.04042) and implemented it for my Stanford CS 229 (Machine Learning) project first - the hack came later. The ObjectCropBot hack allows only cropping and not clicking because it was faster to hack in that, but I think it is ideal to allow both cropping to constrict around target object and clicking (clicking alone leaves too much ambiguity , eg do you want to crop the person, or their shirt, or a subset of that shirt).