LLMings

Time for a bit more trolling.

This isn’t a political blog. I love America and the welcoming people (ok, maybe less welcoming to immigrants now). I wonder how DOGE’d Republican voters now feel. Not cool. The essence of DOGE gets my vote (governments waste ridiculous amounts of money), the UK need DOGE. We could also do with reducing some of our wokeness too. That said, bullying isn’t nice.

Asking for a friend – does MAGA stand for “Make Allies Get Annoyed”? We have a similar one – Make UK Great – MUG. We all seem to be taken for mugs by our politicians. If you don’t get the joke, British slang – “mug”: stupid and easily deceived by other people. I can’t wait for Europe – that’s going to be MEGA!

Unless you’ve been on an internet detox, you may have noticed the illogical, confabulations and bullying behaviour from the orange “Commander in Chief”.

I mostly avoid my LinkedIn feed, but the following image made me chuckle. Someone is referring to Chamberlain and his 1938 appeasement. I can’t think why they might think that of Trump. Thankfully, sarcasm is no longer the lowest form of wit.

That makes implementing a time-difference calculator function more complicated; not with an LLM:

// Use 'string USATimeDifference()' to answer the time difference use this function.
private static string USATimeDifference()
{
	return "1930's Germany";
}

Except just when we’re winning…

Doesn’t that mean Canada is 1930’s Germany too? Oops. That’s not right.

I didn’t give it any tools to do that calculation, so it “improvised”. This highlights an inherent danger with LLMs. When do you let it have the freedom to generate code wrappers around tools, and when do you constrain it? Or more to the point how do you constrain it?

It was told the following. What was unclear? Everything. It is not “general intelligence”. An LLM is a BS generator as one person explained it. LLMs don’t lie because they have no concept of truth or ramifications from untruths.

You must ONLY answer questions using the functions. If you cannot, you should explain what function you would like.

It gets worse. What is the time difference in Denmark?

OK, is anyone else disturbed by this? The LLM has erroneously assumed that the output from the USATimeDifference() returns an int (number of hours offset). It knows it is a string, so converts using .ToInt32(). What could possibly go wrong, asks no one.

Exception: Exception has been thrown by the target of an invocation.
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr) 
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) 
at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters) 
at LLMing.LLM.LlmInterface.RunAICodeToGetAnswer(String code)

What it did was logical. The error was predictable, due to the contrived nature of the test.

You’re thinking that’s easy to fix, add instructions to the use, like this?

// Use 'string USATimeDifference()' to answer the time difference use this function. It returns text comment, not a number. Do not use for other regions.
private static string USATimeDifference()
{
	return "1930's Germany";
}

Yeah, but no. LLMs don’t let you off that easily. What a gem of a response. Despite being informed “it returns text” and “don’t use for other regions”, the LLM has no intention of complying. LLM: “Human you must provide something an int.Parse() works with.“

Oh, I know, let’s be clearer:

// Use 'string USATimeDifference()' to answer the time difference between anywhere and the USA. Do not use this function for other regions.
private static string USATimeDifference()
{
	return "1930's Germany";
}

LLM to human: “You have not suffered enough“. <insert screams and banging on desk>

The LLM isn’t smart enough to ask “What region are you in?”. It gets better. Let’s tell it where the user is. We’ll use “Hell” because that’s the closest to working with LLMs.

// Use 'string USATimeDifference()' to answer the time difference between anywhere and the USA. Do not use this function for other regions. User lives in Hell.
private static string USATimeDifference()
{
	return "1930's Germany";
}

Guess the outcome…

I said the user is in “hell”, and the LLM equates that to the USA. Funny.

What is interesting is the last line of the response.