“What the heck is a search methodology?” I asked, perhaps not using those exact words.
Worse, it would have been useful to have known this before writing and then building the literature review.
A search methodology has structure to it, but like a lot of things, there are different types — in this instance, referred to as a protocol, and common among them is PRISMA.
Having taken the hit like a hero, I sat in darkness to reflect on this illuminating experience. What followed was a conversation with the AI to understand to what degree the Under Cloud adhered to PRISMA.
Somehow, without having any prior knowledge of these protocols, I’d created something that was a good match. I attribute much of this adherence less to luck than to the constructive conversations I’ve had with AI in the past that guided me in this direction. As explained in “Soft pivot” a lot was added, but also some parts of the foundation underwent sympathetic change.
As a learning exercise, the gain line was moved forward (a sporting term, but also — and not wanting to patronise — a learning exercise to those not conversant).
I used the newfound understanding to build a plan of action, to tighten the adherence with PRISMA and the search methodology in general, to be implemented in the near future. But, it was also obvious to me that had the Under Cloud not already supported much of what PRISMA recommended, the remedial activities would have been enormous.
I was also struck by how involving the search methodology is — put simple, a search methodology is an audit trail of activities, such as performing searches across things like databases, the web (in general, but also places like Google Scholar, JSTOR, PubMed and so on), search terms and keywords, and products such as the Under Cloud.
I couldn’t help but make a connection between with what Jose Gomez had discovered, as discussed in “Cognition conundrum” and what had been revealed to me by several professional researchers, and the somewhat involving nature of search methodology as a thing to perform.
What if the search methodology could be used to confound those using AI to write entire dissertations? In effect, make cheating difficult. As the AI explained in its critique of the idea:
A sufficiently motivated student could script the activity — performing searches via the API, linking assets to claims, and generating authentic-looking “discoveryContext” records [a structure designed to contain the aforementioned audit trail]. This is substantially harder than prompting ChatGPT, and it ironically requires the type of technical competence that would make the student capable of doing the actual research. The countermeasure should be framed as “raises the cost of fabrication to the point where doing the work is easier” rather than “prevents fabrication.”
As a software engineer, this is like the salt value used to generate a unique password, to make them difficult to guess — not impossible, but improbable.
But this question also had the shape of something that looked a lot like unique selling proposition.
Imagine an entire academic department requiring the use of the Under Cloud, and to include a search methodology section in their work (not always applicable, so we’ll make a few assumptions about the types of academic department). In addition to the confounding effect of the search methodology, the would-be cheat would also have to dismantle the dissertation (as an example) and place the resulting pieces into their respective claims, along with the corresponding evidence taken from notes and DOI sources.
Let’s be objective:
- Would this prescriptive approach of hoops and hurdles be seen as interference?
- If perceived as hoops and hurdles, we’re talking about friction, in user experience parlance.
A recurring theme of the conversations I’ve had with professional researchers has been the shared concern that the use of AI could erode the understanding of how to use language not least the obvious vehicle of communication but also as the actual apparatus of structure to demonstrate understanding.
Am I onto something? Perhaps … I don’t know, I’m making this up as I go along!
Novel though these idea are, not a single iota of it helped me with the predicament of not having a damn search methodology in the literature review…