F'innews: Winter Wisdom
January 28th: Join us next week in SF, a complicated relationship with discrete choice, and inside the F’inn AI Lab (aka FAIL) (~5 minute read)
Join us for our first meet-up of 2026! We’re hosting another meet-up in San Francisco February 3rd! This event will stir up some conversations that are top of mind for so many of us:
How will AI shape our industry?
How can we define our roles in a way that leverages AI without giving up the humaness of what we do?
No lectures, no slides - just thoughtful discussion with people navigating the same questions you are.
Come for the a free round of drinks, stay for the comradery.
If you have issues with any links, try opening this newsletter in your browser.
If you’re not in the Bay Area, stay tuned as we have plans for meetups in NYC (later February) and other locations coming soon!
Discrete Choice, I love you, but you’re bringing me down
Inspired by one of my favorite bands, LCD Soundsystem, who’s lead singer, James Murphy, wrote a song called “New York, I Love you But You’re Bringing Me Down.” Often played at the end of their set, is a brilliant sign off about a city that he is in love with and at the same time utterly frustrated by its change and gentrification.
In an odd research nerd way, I feel the same about discrete choice (DC), so strap in and let me tell you why.
I’ve spent a good part of my professional life designing and analyzing discrete choice models, and they are some of the best jobs I can recall working on. It’s a great marriage of complexity and efficiency that any other solution forces you to make tradeoffs (hah) to deliver a similar outcome. It’s not surprising that we still get several requests to execute them.
Unfortunately, the thinking around when and how we deploy them is severely outdated and needs to change.
As survey taking has transitioned mostly to mobile, DC tech really hasn’t caught up with being that mobile friendly. For simple designs, it’s still manageable, but for more complex ones, it’s a nightmare.
And then there’s the human attention span that has shrunk from 2.5 minutes in 2004 to just 47 seconds on average in recent years1 and that’s a big problem for respondent engagement in surveys and especially for repetitive tasks of large discrete choice surveys (thanks for sticking with me this far btw).
We started to see cracks form in big DC design data in the last 6 or so years. The most obvious was how long it took to calibrate a reliable discrete choice model. Over the past few years, more complicated DCs have taken extensive calibration to get them to even remotely make sense.
We suspect that when respondents are faced with a large and complex discrete choice exercise, they:
Feature hunt: They’re overwhelmed by the design and therefore focus on just 1 or 2 features that matter to them – selecting whatever option has these available by ignoring the rest. This can inadvertently inflate interest for other features within the configuration that really have zero appeal in real life.
Pick whatever they see on the screen first: If it’s a large design with many SKUs, it’s unlikely many scroll to see the whole thing, causing a large amount of choices to cluster to a base SKU (which are often shown to the left most part of the design).
Complete disengagement: Depending on how long the survey is and how much time it took to get the exercise alone, categories that are not engaging, with too big of a design or wordy features cause many to go straight catatonic in the survey. They’ll click whatever to get through it and get paid, giving you completed surveys with tons of contradictions that will make you scratch your head. Survey platforms monitor straight lining but it’s more challenging to do this within a discrete choice model (particularly ones that are ported from another program).
Beyond simplification, there are many tools at a researcher’s disposal to get creative with approaching DC asks through a different lens (methods like multi-scenario concept testing, maxdiff + price elasticity, etc). In order for them to be successful, you need an open mind and to understand that while you can take the easy way and simply run a complex discrete choice, you’re doing your business a disservice.
To read what to avoid when actually creating a DC model and Brian’s full post, click here!
The Importance of FAILure
At the F’inn Artificial Intelligence Lab (FAIL), we embrace failure as our greatest teacher, particularly in AI, which is evolving so damn quickly. For the past seven years, our lab has operated on the idea that experimentation, even when it doesn't generate any immediate successes, is necessary for creating truly reliable AI solutions for our clients. We don’t chase every AI trend or recommend half-baked solutions just because they’re novel. But we do have a high bar for validity and effectiveness, constantly building, testing, and iterating until we can confidently deliver tools that enhance our clients’ capabilities and insights.
Some of our lab’s biggest failures include finding agentic AI too brittle, persona fine-tuning experiments that prove hollow, synthetic quantitative data that doesn’t match human data, and poor AI-based emotion detection in videos. But we learn, iterate, and keep trying. So our current experiments embrace cutting-edge technologies like RLHF (Reinforcement Learning from Human Feedback) for better persona experiences (and funnier jokes), synthetic quantitative data augmentation, digital twin creation for consumer behavior modeling, agent swarms, and embodied intelligence. Each experiment teaches us something valuable about what works, what doesn’t, and what shows promise for the future as the field of AI develops.
With this methodical approach, we have already created AI products that show our commitment to practical innovation over flashy demos. F’innSight uses Large Language Models to create more interactive, intelligent surveys that can code open-ended responses in real-time and adapt questioning based on participant answers, delivering better and deeper data than the usual static surveys. Our PersonafAI tool recreates detailed personas and buyer segments using synthetic respondents that can simulate almost any type of buyer based on human quantitative data, transcripts, and psychographic measures. And behind these successes lie many experiments, from our CARLIN humor generation system using AI agents that significantly improved funniness scores to our ongoing work with synthetic respondents and digital consumer twins. We believe that with AI moving so quickly for market research, the companies that will ultimately succeed are the ones that combine experimentation with validation, those willing to fail fast, learn faster, and only use solutions that truly serve their strategic needs.
Follow along with us and our FAILures at F’inn.
