"From Foundations to GPT in Text Classification: A Comprehensive Survey on Current Approaches and Future Trends", the latest review article from FnTIR www.nowpublishers.com/article/Deta...

Are search engines getting worse—or is it time to rethink how we search? ADM+S researchers Oleg Zendel, Ashwin Nagappa & Johanne Trippas share insights on search quality, AI & what it means for how we find trustworthy information online @olegzendel.bsky.social @ashwinnag.bsky.social bit.ly/3F8sx44

Is Google search getting worse? Maybe what we want from it has changed

Has the simple search become less useful in a world of low-information, SEO-optimised sites, and often faulty AI summaries? Are there better ways to navigate the web?

bit.ly

Excellent presentation by @admscentre.org.au @rmitcomputing.bsky.social master’s student Kun Ran m at #chiir2025 @marwahalaofi.com @iroldie.bsky.social www.damianospina.com/publication/...

Our @theconversation.com article discussing 4 ethical models of search engines is out! theconversation.com/what-makes-a... @admscentre.org.au @rmitcomputing.bsky.social @umbrellacorpn.bsky.social

What makes a good search engine? These 4 models can help you use search in the age of AI

To make sure search engines serve us well, it’s helpful to imagine these tools having different roles – whether it’s a “librarian” or a “teacher”.

theconversation.com

"Two Heads Are Better Than One: Improving Search Effectiveness Through LLM Generated Query Variants", preprint is up marksanderson.org/publications... Awesome work by @rmitcomputing.bsky.social Masters student @rankun203.bsky.social with Marwah Alaofi and @damianospina.com Presented ACM CHIIR.

For the first time, Nature publishes a list of "retraction hotspots": academic institutions where a great many papers have been withdrawn post publication. www.nature.com/articles/d41...

Exclusive: These universities have the most retracted scientific articles

A first-of-its-kind analysis by Nature reveals which institutions are retraction hotspots.

nature.com

The TREC website has been updated for the first time in... decades? Looking good. trec.nist.gov/index.html

"Illusions of Relevance: Using Content Injection Attacks to Deceive Retrievers, Rerankers, and LLM Judges". Inject unrelated toxic content into a relevant passage, an LLM judge still says the passage is relevant arxiv.org/abs/2501.18536 from Manveer Tamber & Jimmy Lin.

Illusions of Relevance: Using Content Injection Attacks to Deceive Retrievers, Rerankers, and LLM Judges

Consider a scenario in which a user searches for information, only to encounter texts flooded with misleading or non-relevant content. This scenario exemplifies a simple yet potent vulnerability in ne...

arxiv.org

"Multi-stage Large Language Model Pipelines Can Outperform GPT-4o in Relevance Assessment" w/@Julian S. @danulah.bsky.social @umbrellacorpn.bsky.social accepted at #webconf2025! 😱🌟 We present an LLM-based pipeline that boosts relevance assessment accuracy through modular classification. #SIGIR2025

Two SIGIR Information Retrieval greats were made ACM Fellows this year: Maarten de Rijke and Justin Zobel. Many congratulations to both! www.acm.org/media-center...

2024 ACM Fellows Celebrated for transformative contributions to computing science and technology.

ACM has named 55 of its members ACM for transformative contributions to computing science and technology. All the 2024 inductees are longstanding ACM Members whose accomplishments were selected by the...

acm.org

Building on four successful workshops in the last few years, it was decided to create the Inaugural SEASON conference of the Search Engines and Society Network, starting in 2025. Submissions, April 30th. Held in Hamburg. easychair.org/cfp/SEASON2025

Thanks @hscells.bsky.social for pointing me to this

Harrie Oosterhuis@harrieo.bsky.social · 2y ago

My colleagues published about pairwise relevance labeling this last year (and even earlier on arXiv): aclanthology.org/2024.finding... Why I personally think it works better, mainly because it's hard to calibrate a pointwise relevance prediction, but a pairwise prediction hardly needs calibration.

Using LLMs for pairwise relevance decisions, I haven't seen that tried before, but it makes perfect sense to try them here. My impression is that pairwise relevance hasn't been used much in the past because of the cost of lablelling.

Doug Turnbull@softwaredoug.bsky.social · 2y ago

Local LLMs might unlock a new age of democratized search relevance. Here I talk about using Qwen 2.5 locally as a local pairwise search evaluator softwaredoug.com/blog/2025/01...

There have been a number of investigations on whether LLMs can replace humans for relevance assessment. I think the evidence is showing LLMs have a strong role, but won't replace. This recent paper from @claclarke.bsky.social & Laura Dietz supports this view arxiv.org/abs/2412.17156

LLM-based relevance assessment still can't replace human relevance assessment

The use of large language models (LLMs) for relevance assessment in information retrieval has gained significant attention, with recent studies suggesting that LLM-based judgments provide comparable e...

arxiv.org

What a team of keynote speakers. I must confess seeing that Steve Robertson will be there is a thrill. One of the legends of information retrieval reflecting on the field. #sigir2025 sigir2025.dei.unipd.it/keynote-spea...

"Two Heads Are Better Than One: Improving Search Effectiveness Through LLM Generated Query Variants", short paper accepted #chiir2025. Led by our Masters student Kun Ran with Marwah Alaofi, myself, and @damianospina.com. Awesome result Kun! @admscentre.org.au @rmitcomputing.bsky.social

We invite PhD students to submit your work and join us at PhD Symposium @TheWebConf 2025, (www2025.thewebconf.org/phd-symposium). The submission due date is 18 Dec, 2024 (AOE)! We will see you in the beautiful and amazing Sydney down under! #WWW2025 #WebConf2025

International World Wide Web Conference 2025 ( WWW2025 | The Web Conf 2025 )

28 April - 2 May 2025

www2025.thewebconf.org

@iroldie.bsky.social · 2y ago

We invite contributions to PhD Symposium at #TheWebConf 2025. PhD students are encouraged to submit ongoing research related to broad topic of the World Wide Web. www2025.thewebconf.org/phd-symposium Co-chairing this with Gabriella Pasi and Ruihong Qiu

Thanks to the #sigirap2024 people for giving us a mention on the conference bag, brought a smile to our faces.

So many aspects of evaluation including fully synthetic test collections, loved it.

Damiano Spina@damianospina.com · 2y ago

Excellent keynote by Peter Bailey from Canva about how to build a more cost-effective online & offline experimentation framework #sigirap2024 #melbourne #naarm @admscentre.org.au @rmitcomputing.bsky.social

I enjoyed attending the thought provoking two day gathering that helped drive the creation of this document. I look forward to reading it. "Future of Information Retrieval Research in the Age of Generative AI" arxiv.org/pdf/2412.02043

In her talk, @katecrawford.bsky.social delved into the history of Large Language Models, she rightly highlighted the pioneering work of Fred Jelinek who kicked off language models in 1972, however, I wouldn't be faithful to my handle (IR Oldie) if I didn't highlight some IR history...

Just in case you didn't see this shared elsewhere. Maarten writes "I am asking for donations because our daughter Emma passed away". epilepsie.digicollect.nl/maarten-de-r...

Maarten de Rijke kan het niet alleen. Help jij mee?

Epilepsie verstoort levens! Steun je mij met een donatie in mijn collectebus? Met de opbrengst kunnen we samen op zoek naar de oplossing van de verstorende aanvallen. Ik ben je eeuwig dankbaar!

epilepsie.digicollect.nl

What a magnificent communicator @katecrawford.bsky.social is. Loved her talk at the State Library of Victoria on Thursday broadening the perspectives we have about AI. The non-techy who accompanied me was buzzing about this event.

And we have a #Dagstuhl Report! Evaluation Perspectives of #RecSys. Edited by @christinebauer.bsky.social, @evazangerle.bsky.social, and myself. Written by a whole host of fantastic #recsys people. Too many to mention (pic here www.dagstuhl.de/en/seminars/...) drops.dagstuhl.de/entities/doc...

A cool blog post from Ellese Cotterill on her work at Canva to spin up a known item test collection from scratch generated entirely from an LLM. www.canva.dev/blog/enginee...

How to improve search without looking at queries or results - Canva Engineering Blog

How we improved Canva’s private design search while respecting the privacy of our community.

canva.dev

"Metamorphic Evaluation of ChatGPT as a Recommender System" a cool short paper from an @RMITComputing student, Madhurima Khirbat, showing how superficial changes in prompts can result in substantial change in #gpt #recsys style output @admscentre.org.au arxiv.org/abs/2411.12121

Metamorphic Evaluation of ChatGPT as a Recommender System

With the rise of Large Language Models (LLMs) such as ChatGPT, researchers have been working on how to utilize the LLMs for better recommendations. However, although LLMs exhibit black-box and probabi...

arxiv.org

We invite contributions to PhD Symposium at #TheWebConf 2025. PhD students are encouraged to submit ongoing research related to broad topic of the World Wide Web. www2025.thewebconf.org/phd-symposium Co-chairing this with Gabriella Pasi and Ruihong Qiu

International World Wide Web Conference 2025 ( WWW2025 | The Web Conf 2025 )

28 April - 2 May 2025

www2025.thewebconf.org