multi-sensory tagging
cjmconnors — Tue, 03/24/2009 - 15:06
[Note: making a private thread public.]
I want to be able to tag things by virtual representations of sensory inputs: colors, textures, sounds, tastes, smells, emotions. Need to be able to do on mobile as well to incorporate real-world experiences and tag them this way at the time of their happening to capture the full essence/context.
Would also be cool to add things like the lat/long (GPS data), weather-data for where/when a pic was taken.
Could be neat to add info on "what's playing" to a tagged item. (Shazam? Verizon or Blackberry song ID tools?)
adding context(s)
Kevin — Sun, 07/20/2008 - 23:41Add to geo location: speed, direction, mapping to be able to suggest, "I was going south on I-10 that Friday afternoon" and be able to put that itself in context. For instance, "It was Friday afternoon, and just 45 minutes after I twittered, 'insert-twitter-text-here'." And omg I love the adding "what's playing" to the tagging. This is so unbelievably simple, and oh-so-smart multi-sensory indexing! (I love the weather idea too.)
Then we could consider semantic proximity, and if that seems overwhelming (or we can learn to tell when it's just plain wrong), we could do TF/IDF. We could consider time/location, thesauri or taxonomy or ontology branches.
I want to capture texture (CC made me think of this in one of her synesthesia descriptions).
Remix data
christine — Mon, 07/21/2008 - 11:58The obvious thing - which I'm posting here just to capture and avoid a DUH moment - is to pull in open data from sources like Wikipedia, Flickr, and user-authenticated sources that are close semantic matches. An ontology won't be enough - an index itself is interesting only to a select group of infosci uber-geeks. See Vicinit (European-focused iPhone 2.0 app).
Excellent, now I have a reason to upgrade to the 3G... ;) Get your phone Lynch!
adding emotional context
Kevin — Sun, 07/20/2008 - 23:43And emotions?!? Are you kidding me? Ok, so let's go *way* out here: if we're going to be wired in, then why not get the polygraph-equivalent for the person and use that as part of our index? We could represent this, but the key would be the user recognizing the representation for the memory. (So, "My heart rate was elevated, I was sweating." - How do we 'say' this?) This could evolve...and far. If we can just show the cross-sensory indexing approach works, then we could go out to an MIT, get their wearable whatevers, and plug their stuff in. Then we play it back through their little monocle (praying that the visual representation captures the cross-sensory stuff), and... omg.
subvocalization as a capture and retrieval command
Kevin — Sun, 07/20/2008 - 23:51Ok, so, if we're *there* then we might as well be able to capture subvocalizations (did I mention MIT?) as both the capture and retrieval mechanism. It may be the closest we can come to a command language that is not our brains. So, we literally utter, "Capture" and our set of contexts is captured. Utter "Recall " and we go out and get our (now-patented) representations of related contexts.
If we don't get "there" anytime soon, it won't matter if we've gotten the basis for the cross-sensory mappings. We will be taking all the digital context (what's on the screen), and the physical context (what's outside the screen, including what we're feeling/sensing). Provide the mappings, and someone else can put on the front and back ends.
Jott
christine — Mon, 07/21/2008 - 09:35Similar to how simple it is to Jott - I think there was a project once upon a time where we were talking about voice blogging in the field with speech-to-text transformation and then analysis. But it would be interesting to combine all of this on something like Swurl's timeline: http://cjmconnors.swurl.com/timeline; or take Swurl one further and plot a map. Could be a fun "Tufte" project!
Brings up a host of socio-political implications.
wearables as context
Kevin — Mon, 07/21/2008 - 00:05So all the dang RFID tags are being put into clothes, so the answer to "What was I wearing?" becomes possible!!! And, heck, if it's RFID, and we're in a social situation, "What were *they* wearing?"
Add relative position, over time, and all of a sudden you've got a virtual representation of movement. It would be like when a camera is circling around you. You sample at certain points, and then interpolate to get the movement. Put a person's picture in the clothes, and then move through the recreated context a la Photosynth.
a little creepy, but
christine — Tue, 08/05/2008 - 11:52i's not just wearables... think about a dog that's been chipped. We could associate a smooth-coated dog with the tag smooth, or soft or fluffy. A wiry-haired dog could be "scratchy" or rough, etc.
People are getting chipped; could a person be associated with tags of the users emotional reactions to that person?
Could a tangible be associated with the users reaction? Traffic accidents waiting to happen if someone were driving down the road and had random shipments trigger emotion tags on their personal feed...
elaborate, please - emotion tags, tangible -> reaction
Kevin — Sat, 08/09/2008 - 00:45"...with tags of the users emotional reactions" - who's being tagged and who's doing the tagging here?
"...a tangible...with the users reaction" - same
from our user's perspective
cjmconnors — Sat, 08/09/2008 - 20:00OK, so, let's say you use BrightKite or Loopt or some kind of location-based or RFID-like tool. And let's say you have a Chumby-like app on your mobile which is sync'd to your FriendFeed. You walk into one of your favorite restaurants, and your tag cloud starts displaying things like "asshole" "anger" "resent" as well as "sinful" "yummy" and "best friend" "dancing" "shopping." Why get all those various things? Because you're with one of your best pals, with whom you like to shop and go dancing, and you tagged your friend with these labels when you added her to your social network somewhere (say, via OpenSocial.) Not 5 feet from you is the dessert cart, with a molten chocolate cake, which while posting a great recipe you found for it to your del.icio.us feed you tagged as "sinful" and "yummy." Then, you look around the dining room, and see your ex, the lying, cheating bastard who broke your heart a year back. Got the idea?
Now, the long term would be the ability to have your own chip trigger a physical/emotional reaction, but that would be a bad idea; it makes me think of the Star Trek movie when Data first got his emotion chip and became paralyzed. Don't think I'd want something triggering any more emotions in me than I already have!
Or to take it further, if we add the trust bit, you can be notified if there are people in the room who are in your friends trust networks - let's say you're at a party where you don't know a lot of people. It could be like LinkedIn for the party - and, depending on your mood that night, you can gravitate towards the people your intellectual friends hang with, or your work friends, or folks like the crazy people you hung with in college. You also now have a way to start a conversation more meaningful than "Hey, I see you're on BrightKite too."
Again, social implications here for non-professional environments, but no worse than anything that already goes on.
taking it yet further, and out of my sphere of knowledge
cjmconnors — Sun, 08/10/2008 - 09:53Connect this to a human-interface device - a monitor for heart rate, blood pressure etc - in a bracelet or watch (been done before) to capture the physical manifestation of an emotional or intellectual reaction, to include that with the time and geospatial data. Think of the applications for medical research; rather than forcing the user to stop and journal certain metrics, or to lug around equipment, have it all added WITH the bonus of having it in a fuller context or their physical and emotional states - with a link to the history of the interactions -- how we behave in certain situations or around certain people. Could be as simple as tracking down an allergy or as complex as identifying a trigger for multiple personality disorders.
Yikes, I'm out of my league here... Want to chime in?
Post new comment