Friday, April 14, 2023

Watching ChatGPT explain a reasoning problem is like watching someone who learned the language phonetically take an improv class

When trying to keep your head above the hype and bullshit surrounding large language models, here's a handy trick. Ask yourself how many people have answered exactly the same question in print over the years. If the answer is a large number, it would not be surprising if the LLM approach of looking at the likelihood of certain words and phrases appearing with other words and phrases actually produced something that seemed to show comprehension and awareness. 

For example, if you asked ChatGPT to prove some familiar theorem, there's a good chance that the algorithm would stitch together an acceptable answer out of the thousands of explanations out there. 

 


It may look impressive, but it no more demonstrates an understanding of mathematics than reciting a passage from Goethe learned phonetically demonstrates a mastery of German grammar and syntax.

Here's what it looks like when an LLM encounters a new problem.







It is not just that the answer is wrong or that the explanation is wrong; it's that they seem unaware not just of the problem, but of each other. The answer at the top is different than the answer at the bottom and neither matches the steps given. 

We need to start thinking of these systems not as generative AI but as regurgative. In some cases that's good enough, niche applications like generating unimportant boilerplate text or writing code snippets, but even in those limited area, it can only function where humans have not only explored a question, but have done so in such exhaustive detail that the algorithm can autocomplete its way to a useful response.

Thursday, April 13, 2023

Thursday Tweets -- you can skip all the snarky stuff after the first tweet, but you have to watch this segment on the Tennessee legislature

This has been building for years.

 This one is sleazy in a particularly telling way.




And keep an eye on this part of the story.

More fun with Ron and Don




And humiliated him. Don't leave that out. 

Most historians say droit du seigneur was a myth, but for this Mar-a-Lago visitor...


You know what's fun in a schadenfreude sorta way? Remembering all the wise pundits downplaying the impact of Dobbs. 


 

 In a shocking development, John Yoo was able to justify Clarence Thomas's behavior.




And while on the subject of Nazis.


 Watching these people convince themselves that Joe Biden is a wild-eyed leftist is still amusing more than two years in.

 
For those on the left still supporting Russia in the name of anti-imperialism.

Checking in with Elon




 

 On a related note, getting people to stop driving vehicles that are way bigger than they need would probably do more than any recent proposal pushing EVs.

 

This week in AI hype.



 

And finally a visit to the nerd corner.










And for the Mad fans who are code nerds.

Wednesday, April 12, 2023

Twelve years ago at the blog -- {2,2,4,4,9,9}, {1,1,6,6,8,8}, {3,3,5,5,7,7}

Warren Buffett had a set of non-transitive that he used to challenge people's intuitions about probability. He once challenged Bill Gates to a game. Gates looked at the dice and said, "sure, you go first."

Monday, April 18, 2011

Cheap Beer, Paradoxical Dice, and the Unfounded Morality of Economists

Sometimes a concept can be so intuitively obvious that it actually becomes more difficult to teach and discuss. Take transitivity. We say that real numbers have the transitive property. That means that if you have three real numbers (A, B and C) and you know A > B and B > C then you also know that A > C.

Transitivity is just too obvious to get your head around. In order to think about a concept you really have to think about its opposite as well --

A > B, B > C and C > A. None too imaginatively, we call these opposite relationships intransitive or non-transitive. Non-transitive relationships are deeply counter-intuitive. We just don't expect the world to work like that. If you like butterscotch shakes better than chocolate shakes and chocolate shakes better than vanilla shakes, you expect to like butterscotch better than vanilla. If you can beat me at chess and I can beat Harry, you assume you can beat Harry. There is. of course, an element of variability here -- Harry might be having a good day or you might be in a vanilla kind of mood -- but on average we expect these relationships to hold.

The only example of a non-transitive relationship most people can think of is the game Rock, Paper, Scissors. Other games with non-transitive elements include the boomer classic Stratego where the highest ranked piece can only be captured by the lowest and my contribution, the game Kruzno which was designed specifically to give students a chance to work with these concepts.

While these games give us a chance to play around with non-transitive relationships, they don't tell us anything about how these relationships might arise in the real world. To answer that question, it's useful to look at another game.

Here are the rules. We have three dice marked as follows:

Die A {2,2,4,4,9,9}

Die B {1,1,6,6,8,8}

Die C {3,3,5,5,7,7}

Because I'm a nice guy, I'm going to let you pick the die you want. I'll then take one of the two remaining dice. We'll roll and whoever gets the higher number wins. Which die should you pick?

The surprising answer is that no matter which one you pick I'll still have the advantage because these are non-transitive dice. A beats B five out of nine times. B beats C five out of nine times. C beats A five out of nine times. The player who chooses second can always have better odds.

The dice example shows that it's possible for systems using random variables to result in non-transitive relationships. Can we still get these relationships in something deterministic like the rules of a control system or perhaps the algorithm a customer might use to decide on a product?

One way of dealing with multiple variables in a decision is to apply a threshold test to one variable while optimizing another. Here's how you might use this approach to decide between two six-packs of beer: if the price difference is a dollar or less, buy the better brand; otherwise pick the cheaper one.* For example, let's say that if beer were free you would rank beers in this order:

1. Sam Adams

2. Tecate

3. Budweiser

If these three beers cost $7.99, $6.99 and $5.99 respectively, you would pick Tecate over Bud, Sam Adams over Tecate and Bud over Sam Adams. In other words, a rock/paper/scissors relationship.

Admittedly, this example is a bit contrived but the idea of a customer having a threshold price is not outlandish, and there are precedents for the idea of a decision process where one variable is ignored as long as it stays within a certain range.

Of course, we haven't established the existence, let alone the prevalence of these relationships in economics but their very possibility raises some interesting questions and implications. Because transitivity is such an intuitively appealing concept, it often works its way unnoticed into the assumptions behind all sorts of arguments. If you've shown A is greater than B and B is greater than C, it's natural not to bother with A and C.

What's worse, as Edward Glaeser has observed, economists tend to be reductionists, and non-transitivity tends to play hell with reductionism. This makes economics particularly dependent on assumptions of transitivity. Take Glaeser's widely-cited proposal for a "moral heart of economics":

Teachers of first-year graduate courses in economic theory, like me, often begin by discussing the assumption that individuals can rank their preferred outcomes. We then propose a measure — a ranking mechanism called a utility function — that follows people’s preferences.

If there were 1,000 outcomes, an equivalent utility function could be defined by giving the most favored outcome a value of 1,000, the second best outcome a value of 999 and so forth. This “utility function” has nothing to do with happiness or self-satisfaction; it’s just a mathematical convenience for ranking people’s choices.

But then we turn to welfare, and that’s where we make our great leap.

Improvements in welfare occur when there are improvements in utility, and those occur only when an individual gets an option that wasn’t previously available. We typically prove that someone’s welfare has increased when the person has an increased set of choices.

When we make that assumption (which is hotly contested by some people, especially psychologists), we essentially assume that the fundamental objective of public policy is to increase freedom of choice.


But if these rankings can be non-transitive, then you run into all sorts of problems with the very idea of a utility function. (It would also seem to raise some interesting questions about revealed preference.) Does that actually change the moral calculus? Perhaps not but it certainly complicates things (what exactly does it mean to improve someone's choices when you don't have a well-ordered set?). More importantly, it raises questions about the other assumptions lurking in the shadows here. What if new options affect the previous ones in some other way? For example, what if the value of new options diminishes as options accumulate?

It's not difficult to argue for the assumption that additional choices bring diminishing returns. After all, the more choices you have, the less likely you are to choose the new one. This would imply that any action that takes choices from someone who has many and gives them to someone has significantly fewer represents a net gain since the choice is more likely to be used by the recipient. Let's say we weight the value of a choice by the likelihood of it being used, and if we further assume that giving someone money increases his or her choices, then taking money from a rich person and giving it to a poor person should produce a net gain in freedom.

Does this mean Glaeser's libertarian argument is secretly socialist? Of course not. The fact that he explicitly cites utility functions suggests that he is talking about a world where orders are well defined, and effects are additive and you can understand the whole by looking at the parts. In that world his argument is perfectly valid.

But as we've just seen with our dice and our beer, we can't always trust even the most intuitively obvious assumptions to hold. What's more, our examples were incredibly simple. The distribution of each die just had three equally probable values. The purchasing algorithm only used two variables and two extremely straightforward rules.

The real world is far more complex. With more variables and more involved rules and relationships, the chances of an assumption catching us off guard only get greater.



*Some economists might object at this point that this algorithm is not rational in the strict economics sense of the word. That's true, but unless those economists are also prepared to argue that all consumers are always completely rational actors, the argument still stands.

 

Tuesday, April 11, 2023

Canada's growth

This is Joseph

There is a long conversation right now on whether Canada is broken or if Canada is not a serious country. I was talking about this with Mark and he was skeptical so I decided to lay out the numbers. One item in evidence is long term GDP change compared to the USA and Australia.

Here are some numbers

Canada (Year, GDP,  GDP per capita, population)

1990 $593.93B $21,448      28,347,641

2021 $1,988.34B $51,988      38,155,012


Australia (Year, GDP,  GDP per capita, population)

1990 $311.43B     $18,250    17,048,003

2021 $1,552.67B     $60,443    25,921,089


USA (Year, GDP,  GDP per capita, population)

1990 $5,963.14B $23,889    248,083,732

2021 $23,315.08B $70,249    336,997,624


The striking comparison over the past 30 years is Australia. Australia had a 52% increase in population and managed to jump ahead in per capita income despite being behind when I graduated high school. Yes, in the old days when the mighty cave person hunted the fierce dinosaur, Canadians were richer than Australians. 

Australia went from 52% of the GDP of Canada to 78% of the GDP of Canada over the next 31 years. Canada grew 35% in population, an easier pace to sustain, and still managed to have slower growth in both total and per capita GDP. The ability to boost per capita GDP at the same time as rapid population growth is a really neat trick if you can manage it. 

The USA isn't a great comparator, either. At the start of this time period the USA was 11% richer per capita, while at the end it was 35% richer per capita. US GDP grew 391% despite the same % population growth as Canada, while Canada managed 335%. That gap is the source of the growing divide in per capita income.

More recently, it looks like Canada is adopting a high immigration strategy. Canada's strategy looks a lot like the Australian plan. Obviously the Australians are doing something right and it would be smart to emulate them. But the worry is that we are slowly losing ground and that speeding up immigration in a time of relative decline compared to peer nations makes it harder to build social consensus. A brisk relative increase in wealth is a great way to sustain public support for high immigration  -- people enjoy becoming more affluent. It remains to be see if Canada can turn this sluggish growth trajectory around and make this new plan work.

That's the real test as to whether we are still functional.  

P.S. The best follow-up to the broken versus serious is here. The inspiration is here

The inspiration included New Zealand which seems designed to make Canadians sad with 550% growth in GDP over the same time period and an increase from 64% of Canada's per capita GDP to 94% with similar population growth to Australia:

New Zealand (Year, GDP,  GDP per capita, population)

1990 $45.50B         $13,663        5,129,727

2021 $249.89B $48,781        3,397,389

The huge immigration surge of the late Trudeau administration (really the last few years) has not really led to the sort of boom that the pacific part of the Anglosphere are experiencing. The UK is hard because of Brexit. 

Another interesting factor is child-bearing. Canada's most recent year (2020) is 1.4 children per woman despite a large immigrant population. In contrast, it's peers are all 1.6 (Australia, NZ, and the US). So it has to struggle harder against the demographic headwinds. Canada is closer to Japan (1.3) than to the US (1.6). 

Monday, April 10, 2023

"I'm Dreaming of a White Easter"



I went up to the Sierras this weekend and finally checked snowshoeing off my to-do list. It absolutely kicked my ass -- trudging through deep snow, often using muscles that hadn't seen any action in forever, all at around 7,500 feet (not the highest peak around here but high enough). Saturday was beautiful and the views were stunning and I'm looking forward to getting an earlier start next season.

As always, California is a study in contrasts. The cottage I rented was four hours from LA, and the drive through the Central Valley was warm and sunny. It's supposed to hit 90 degrees in Bakersfield today, but you'll still be able to see snow covered slopes from the 99 for at least a few more days.

The concern now is the coming snow melt and the risk of more floods, particularly in the Valley, but we are still very grateful for a break in the drought. Not to minimize the dangers, but in the West, too much water is almost always better than too little.

And wet winters bring superblooms.


























Friday, April 7, 2023

Thursday Tweets 2 -- If you're tired of the news, just skip to the end and watch the luckiest man alive




Attorneys usually love high-profile cases, I wonder why more big names haven't jumped at the chance to represent Trump.





Peter Baker remains an embarrassment.





The humiliation of DeSantis continues.

 

Among the mistakes the pundit class made in the wake of Dobbs (and there were many) was underestimating the secondary effects (impact on women who have miscarriages and ectopic pregnancies) and almost completely failing to anticipate the tertiary.


The saner members of the conservative movement (a group I never expected to put Ann Coulter in) see where this is leading; they just can't do anything about it.



Both Sides.
Biden -- Putin wanted the Finlandization of NATO. He got the NATO-ization of Finland.


Always projection.

 

Oh, Elon




We need to blog about this.

And this. And this,

For a while in the lat seventies, all the major drug cartels laundered their money through pet rocks.



To be honest, I'm a little baffled now.

"Could machine intelligence become prophetic?" -- In its current version the odds are low, but given that the NYT predicted a red wave, an unstoppable DeSantis and a crushed Ukraine, the bar isn't that high.







Between this and altruism, "effective" is getting a bad name.

Nothing to worry about here.


Cool.


And in closing.






Thursday, April 6, 2023

Thursday Tweets part 1 -- Cheesy Wisconsin Goodness

Much less of a close call for democracy than we're used to.



"Democrats shouldn't be afraid to run on abortion and democracy" seems like something we should mock for obviousness, but I'm sure some analyst for the NYT is about to advise just the opposite.



This would be where a functional party acknowledges the will of the people and starts to adjust its course.



Clarification or walk-back? You be the judge.

 

And in closing,  the worst concession speech ever. *

* And, yes, I am counting "you don't have Nixon to kick around anymore." At least that had some style.

Wednesday, April 5, 2023

Technical difficulties

Blogger's been playful today.  (Apparently the confirm button is now optional)

My apologies to Popehat (Ken White)

This is Joseph.

One thing that I often used to say was that Canada's limits on free speech showed how you could have a well organized country in which speech was limited. Some recent examples have made me a bit less enthusiastic. Let us start with the big one.

In 2015, an OPP officer was charged with drug trafficking. In 2018 he was convicted  He is still on the OPP payroll to this day, because of reasons shielded by privacy:
This was initiated on Nov. 14, 2018, but the hearings were “delayed multiple times,” OPP spokesperson Bill Dickson said Friday. The process continued until November 2022 when the adjudicator sided with the OPP and ordered Redmond be dismissed.
Citing privacy concerns, Dickson wouldn’t elaborate on the nature of the delays.
He was then charged with sexual assault on Oct 15, 2021 and even now:
. . . is still facing “17 additional serious criminal charges including assault, aggravated assault, assault with a weapon and others in connection with multiple victims.”
What is chilling is how often these failures have been shielded by press blackouts, which reduced the pressure to do something and failed to bring attention to how the Ontario Provincial Police are completely incapable of pursuing consequences for bad behavior.


Meanwhile, the premier of Alberta is threatening a defamation suit. For accurate reporting on what she actually did. I will outsource this one:


Now the issue is that she may have been speaking poorly, a defense raised by Jen Gerson:
Complicating matters is that Smith at this time had an embarrassing habit of publicly conflating Crown prosecutors — ie; "our prosecutors" — with justice department officials. This point was noted by columnist Lorne Gunter; it's therefore not entirely clear whether Smith is telling Pawlowski that she is poking members of her own justice department (which could be appropriate, if ill-judged) or individual Crown prosecutors overseeing COVID files (which would be entirely out of line.) 

But how can it be helpful for the reporting on an actual recording to be a part of a defamation suit? This could so easily have a chilling effect on legitimate reporting and is clearly a point of public interest. 


So  maybe the first amendment, with all of its flaws, has some benefit for government accountability? 



Tuesday, April 4, 2023

2022 was a bad year for conventional wisdom [Trump/DeSantis edition]

August, 2022

 Hear Me Out: Trump Won’t Run Again by Jeremy Stahl writing for Slate.

April, 2023

 DC Insiders and GOPs Wake Up and Smell the Coffee by Josh Marshall

It seems like the whole political world is waking up to the reality that absent some dramatic and unlikely new development, the 2024 GOP primary isn’t just Donald Trump’s to lose, it’s very difficult to come up with a scenario in which he does lose. One new poll illustrates numerically what is clear enough from the news in front of us. In a head-to-head race, A Yahoo/Yougov poll showed Donald Trump jumping to a 26-point lead over Ron DeSantis (57%–31%) from a 8-point lead less than two weeks ago. As recently as February, it was a 4-point lead. In a ten-candidate field — the more real-world scenario — Trump holds 52% support while DeSantis falls to 21%. 

...

Something like this state of affairs may continue for the better part of the year, with Donald Trump dominating the race with no clear and credible challenger. Or we may see another candidate like Virginia Gov. Glenn Youngkin rocket forth like a GOP primary memestock harnessing the same latent Trump-skeptical votes that fueled Ron DeSantis’s rise in late 2022. But those boomlets are fueled by the hope that the chosen candidate might be able to unseat Trump rather than any particular attachment to the candidate themselves. So it can crater as quickly as it rises. Once it becomes clear they can’t unseat Trump, most of the support disintegrates. 

If I have the time, I'll go back and do a deep dive into the Stahl piece, which was awful as a piece of analysis, but was pretty good as a bad example. For now though we'll just leave it up as a reminder that almost all of the mainstream press (the NYT, Slate, Politico, and who knows how many others) were fully invested in the DeSantis Ascendant/Trump in Ruins narrative. We called this out as wishful analytics at the time, making many of the same points we made in 2015 in response to the first incarnation of the "Trump won't get the nomination" narrative.

I really like Marshall's memestock analogy. It captures the bubble mentality around not ______ candidates, from Scott Walker to Ron DeSantis to whoever the press corps' next GameStop is going to be.


Monday, April 3, 2023

A long shot you might want to keep an eye on

I've never voted for a Hutchinson and I probably never will, but I've never underestimated one either.

For around three decades now, the Hutchinsons have been the most powerful political family in Arkansas. I have never called any of them overly principled, but they do tend to be smart, none more so than Asa.

Hutchinson is in a sense doing the opposite of nearly every other prominent National Republican. While almost everyone else is embracing more and more toxic positions in an attempt to out Trump Tump, Hutchinson has for the past few years relied on his solid conservative bonafides to allow him to step back from these unpopular stands while keeping his support from the base. By the admittedly insane standards of 2023, while the rest of the competition has chained themselves to stances that poll in the mid thirties to high teens, he has done a remarkably good job of setting himself up as the reasonable and electable conservative.

Hutchinson is making a long shot bet here, but it's a smart one and he has been pursuing it with a high degree of skill and success. If the fever does break within the next 12 months and Republicans start thinking about the election in terms of viable candidates and broadly popular appeal, Hutchinson will be at or near the top of the list. The base will never forgive Mitt Romney and, barring black swans, the general electorate is highly unlikely to embrace the platform or elusive personal charms of Ron DeSantis, but Asa appears to have hit the sweet spot.

Politico has a recent piece on Hutchinson's old school Republican pitch. The policy sections are well reported. The political analysis is considerably weaker. It buys into the assumption made by pretty much all the candidates except Hutchinson that the GOP can somehow get rid of Trump without losing Trump voters. I've always been skeptical of the party's chances of pulling that one off.

Friday, March 31, 2023

"The Future is a Dead Mall - Decentraland and the Metaverse"

I've found Folding Ideas annoyingly inconsistent. Line Goes Up remains perhaps the definitive overview of the NFT mania, but most of his other videos had left me definitely underwhelmed.

"The Future is a Dead Mall" doesn't reset the high score but it is certainly the second best thing I've seen on the site. The story of Decentraland is wonderfully absurd and rich with telling details about the culture that produced it. The length is a bit daunting (though still shorter than "Line Goes Up"), but there's more than enough content to fill the time. 

To get the full comic effect (and further lower your opinion of the ever credulous NYT), check out this article on virtual real estate from the height of the bubble. (You can find it here. It does not deserve another direct link.)

The Metaverse Group has a real estate investment trust, and it plans to build a portfolio of properties in Decentraland as well as other realms including Somnium Space, Sandbox and Upland. The internet may be infinite, but virtual real estate is not — Decentraland, for example, is 90,000 parcels of land, each roughly 50 feet by 50 feet. Among investors, there’s a sense that there’s gold in those pixelated hills, Mr. Gord said.

“Imagine if you came to New York when it was farmland, and you had the option to get a block of SoHo,” he said. “If someone wants to buy a block of real estate in SoHo today, it’s priceless, it’s not on the market. That same experience is going to happen in the metaverse.”

Last week, Tokens.com closed an even larger land deal in Decentraland’s fashion district for roughly $2.5 million. The company, which says the real estate transaction was the largest in metaverse history, plans to develop the area into a virtual commerce hub for luxury fashion brands, à la Rodeo Drive or Fifth Avenue.

Mr. Kiguel estimates his portfolio in the metaverse is valued at up to 10 times more than its purchase price, and much of the reasoning will sound similar to anyone who has ever bought or sold real estate.

“It’s location, location, location,” he said. “A parcel of land in the downtown core, which has a lot of visitor traffic, is worth more than a parcel of land in the suburbs. There’s a scarcity value.”






Written by Dan Olson and Nathan Landel 

 Produced and performed by Dan Olson 

Thursday, March 30, 2023

Thursday Tweets -- valid until “21 years after the death of the last survivor of the descendants of King Charles III.”


What Does 'Throw Shade' Mean? - Merriam-Webster

"Shade is a subtle, sneering expression of contempt for or disgust with someone—sometimes verbal, and sometimes not."






Another example of the PayPal Mafia's superior intellects.


 And more deep thoughts from Silicon Valley.




As frequently mentioned before, Bonier was the poll analyst to watch in 2022.








Steele has had almost as long and strange a journey as his ex-brother-in-law.



Contagion.


Pepper's whiteboards are more interesting than they ought to be.


We'd all be better off if we start listening more to Magaret Sullivan and Jay Rosen


Finally seeing some attention paid to the grid.





Feral disinformation.



In case there's someone out there who hasn't signed up yet.

Bezzle aftermath from the indispensable Linette Lopez.

LLM dept.


And general nerd stuff.*



* Language nerds would point out that's not the correct useage of 'stuff.'