Bash batch. Converting .avi to .mp4 for import to Davinci Resolve

Bash makes things so easy for doing batch operations on a bunch of files.  I have a bunch of .AVI files from my old camera that Davinci Resolve won't handle.  ffmpeg converts well enough to .mp4 for import.  Since I have a bunch of clips, conversion is just a bunch of command lines:

$ for filename in *.AVI; do filepart=${filename%%.AVI}; if [ ! -e ${filepart}.mp4 ] ; then echo ffmpeg -i ${filepart}.avi ${filepart}.mp4; fi;  done | sh -x

 Formatting and readability is a bit fugly on the command line but this does it nicely.  As a bonus it sees if things are converted already and don't do a conversion.

Telephones

Quick article about how the US senate almost banned rotary phones - https://www.senate.gov/artandhistory/history/minute/Senate_Considers_Banning_Dial_Phones.htm

An old boss of mine had an amusing anecdote about somebody years ago saying that with the enormous growth of telephones in the US everybody would be employed as a telephone operator.  The amusing part of my boss' anecdote was that this person was actually right.  Since we type in phone numbers when we want to talk to somebody, we are in effect working as a phone operator.

Of course in good old 2019, we pretty much have contact apps universally available so we aren't likely to have to type in phone numbers any more.  Unless it's from the menu of the new restaurant that opened and we want to get takeout.

Statistics, lies, whatever.

Every now and then I hear the oft repeated $verylarge% of drivers consider themselves to be above average.  Ha ha, stupid drivers, you suck, you are really bad drivers, you just don't know it or worse don't want to admit it.  After all, being bad is one thing, but being unaware that you are bad, that means you are a general moron as well.

But, if we have a sample of drivers, one rates at a 2, three rate at a 10, the average rating is 3.2.  This would mean that the three scoring at 10, or 75% of the drivers are above average.  Waaaaay above average.

What am I missing here?

On Poverty

Interesting post - https://www.irishtimes.com/culture/why-do-so-many-americans-hate-the-welfare-state-1.4057860

US culture based on Protestant ethics.  European roots were later changed by various world wars to be more supportive of welfare whereas US was largely unaffected by those wars so the ethics lived on.

Professor Elizabeth Anderson notes:

“You know,” she says, “America was so dominantly Protestant for such a long time. We have a substantial number of Catholics but the culture was really shaped by Protestants – in term of their total cultural domination of the United States at its founding, and really continuing.”

One aspect of this, she highlights, is the adoption of a Protestant work ethic as a core value in society. This has a positive side – in honouring human labour – but it also has a negative side.

“There is a profound suspicion of anyone who is poor, and a consequent raising to the highest priority imposing incredibly humiliating, harsh conditions on access to welfare benefits on the assumption you’re some kind of grifter, or you’re trying to cheat the system.

“There is no appreciation for the existence of structural poverty, poverty that is not the fault of your own but because the economy maybe is in recession or, in a notorious Irish case, the potato crop fails.”

Energy

Energy, here on earth, is apparently finite.  We power things with oil and natural gas which is a finite resource held within the bounds of the sphere that we live on.  In the world of the future though, we anticipate getting this energy from the sun, a large glowing sphere nearby.  You could imagine though that solar energy is also finite in the sense that the earth has a finite surface area with which to capture the solar energy.  Could it be in the future that our resource demands are so great that we would admonish people who were wasting solar energy?  Why isn't that square meter being used to capture solar energy or being used to power the photosynthesis for plant xyz?  What a waste, such a shame!

Say what?

Great article from Paul Graham - http://www.paulgraham.com/say.html.  It's from 2004.  It's timeless.  The man is a genius.  Or maybe he is totally wrong and I just agree with him.  Read the article.

Edit:  Interesting somewhat related article I was directed to.  https://slatestarcodex.com/2017/06/21/against-murderism/

Maintenance?

Now this is an interesting read.  http://themaintainers.org/blog/2019/7/30/why-do-people-neglect-maintenance.  I often see a sort of tension between maintaining something and ignoring it and replacing it after a period of time.  It's almost as if we think that one strategy is universally better than the other.  Really we should be thinking about characteristics that make it a better choice in some cases.

Smart coral polyps?

From https://phys.org/news/2019-10-dead-corals-regrow-fatal.html

But the researchers found that in 38 percent of the impacted colonies, the polyps had devised a survival strategy: shrinking their dimensions, partly abandoning their original skeleton, and gradually, over a period of several years, growing back and starting a new skeleton.

Should I actually read that as the polyps that did not shrink died off whereas those that were able to shrink survived and were  able to survive and grow back over a period of years?  Some polyps shrink due to the problems, some just die off?  Guess which ones we see around after a longer period of time?

This is just like the cats that died when exposed to sunlight.  Those all died off and were not able to reproduce and we are not aware of them at all since all we have left are cats that are OK with sunlight.

Never good at lisp

Back in the day when I was learning lisp, I would have a hard time understanding constructs that were just too nested.  I would see things like:

something(somethingelse(trythis(internalpart(updateothers(runstuff)))))

I can see a similar thing happening sometimes with Python where it's easy to keep wrapping object calls together and you end up with hard to decipher code:

something(parameter).object(otherthing(innerpart).state(optionalrewind))

There are some people who talk in similar ways at work and use parenthetical sub clauses before getting to the actual point and I have a hard time following what the actual point is:

We need to figure out, and not necessarily now, but at some point in the near future perhaps when we can get a consensus or when the time is right but not just because we want to be opportunistic agnostic, what, or perhaps even first why or maybe the rationale behind this and related activities that you and according to at least a couple of other people although this is just third party because I haven't heard directly from them but I have been able to glean by reading between the lines of interactions, our next steps should be.

Parsing that out, the real meat of the sentence is (in bold):

We need to figure out, and not necessarily now, but at some point in the near future perhaps when we can get a consensus or when the time is right but not just because we want to be opportunistic agnostic, what, or perhaps even first why or maybe the rationale behind this and related activities that you and according to at least a couple of other people although this is just third party because I haven't heard directly from them but I have been able to glean by reading between the lines of interactions, our next steps should be.

I think as a consumer or target of the conversation I would like to hear the bold part first and can then get the rest of the sentence after.  Or perhaps the rest of the sentence is background and I should get that first.  But I have a heck of a time trying to keep the idea in my head when it's at the beginning, middle, and end, of a bunch of parenthetical remarks.

My point of this whole post (irony, I'm only getting to it now), is that I have a heck of a time reading and understanding news headlines that seem to string together everything into one sentence

[something...] according to police reports via court order leaked by the assistant to the then deputy director who had been allegedly deposed by the now acting associate vice president under executive approval as reported to news personnel by an anonymous tip said the ex chief after hearing reports of this on Thursday from his staff.

What is there some sort of rationing on periods and capital letters that we could not break this up in to digestible sentences?

Well done!

I'm starting to look a bit more into neural networks.  More to come on that later, but in the deeper parts there are matrices and various calculations.  I noticed that the creators of the numpy package did a really great job in making sure that things are displayed in a nice way that makes it easier to see what is going off.  For example, if we create a matrix with:

(python) x=np.array([[1,2,3],[4,5,6],[7,8,9],[10,11,12],[13,14,15]])

when you go to print it out you see:

(python) x
array([[ 1,  2,  3],
       [ 4,  5,  6],
       [ 7,  8,  9],
       [10, 11, 12],
       [13, 14, 15]])

What is really great here is that as the numbers increase in length, the output is shifted so all the columns line up.  Yes, this isn't rocket science genius, but it goes a long way to making things readable and understandable.  In the "computer" world often we just print things out and leave it up to the person to mentally align and figure things out.  I find that people who often do that sort of messy output are people I have to remind that the computer actually works for us, not the other way around.  If we take 10 minutes to make the computer do a bit more work (ex. line up the columns) then it has made things (in this case comprehension) easier for us.  Let the tool do the work for you!

Jumping to conclusions, part 1

Just started reading American Overdose, the Opioid Tragedy in Three Acts.  I'm on the first page, Introduction: An Epidemic Foretold.  It starts out:

Karen Jennings had once been a manager at McDonalds.  Walking home a little the worse for drink, she fell into a creek and broke her back.  The doctor prescribed painkillers to ease her through the pain of recovery.

and then goes on to talk about the spiral of addiction.

So, leaping to conclusions here, it's almost like that second sentence, "the little worse for drink", which is perhaps just glossed over as accidental, could actual be contributing to the condition, namely that Ms. Jennings might have other issues contributing to addiction.

We will see how the book progresses, but I've always wondered why some individuals can apparently safely take opioids and not spiral into addiction.  What percentage of the patients is that, do 50% not become addicted?  75%, 90%?

Another article I have read in the past was related to a study of rats where in a controlled environment rats could self medicate by pressing a button to receive something (cocaine, opioids?) and they rapidly became addicted.  However, the article continued, the rats were in a cage with nothing but the button.  Other rats in a similar experiment were in a cage with rich rat entertainment, wheels, things to play with, etc.  They found those rats occasionally pressed the button to medicate, but not to the level of addiction that the isolated rats did.  The premise of the article was that addiction isn't fully the chemical itself but also the environment.  They drew parallels with the Vietnam war where there was a lot of drug use when in Vietnam due to needing to mentally escape the horrific situation, however when those soldiers returned back to the US, the drug use mostly dried up since they were back in a normal environment.

In Ms. Jenning's case when she was walking home "the little worse for drink" (kudos for not driving BTW) I have to wonder if she was also not in a bad place to start with that also "fed" into the addiction.

Anyway, good read so far, on page 1.  We will see where the book goes.

[Edit *2019-05-14*]
Became curious about why some people become addicted to alcohol and not others.  Google was challenging because a lot of the answers were about dangers of addiction and how to detect, why it is bad, etc.  But I was looking for scientific reasons...

Found this - https://www.healthline.com/health-news/lateral-habenula-responsible-for-alcholism-040814#3

which incidentally had this gem:

“Whether one is going to use drugs or not is a choice based on what the alternative reinforcers are,” he said. “If you have nothing else to do and life is not very pleasant, if you are a social creature—for example, a rat, and you are solo-housed in a boring environment as rats are generally housed for experiments, in that setting it is very easy to get animals to self-administer drugs. If one has them in an enriched environment in which they can socialize, have sex, groom, all those things that rats like to do, then it’s much more difficult to get them to self-administer drugs. It’s much more difficult to induce something that looks like an animal model of addiction.”It’s a fair bet that if humans were forced to live in the same conditions as the rats in Taha’s experiment—alone, in a tiny cage, with nothing to do—and given unlimited access to drugs, they might pass the time by using drugs, too.

They note that genetics may be involved namely that the inability of the lateral habenula region to respond to negative outcomes (painful stimulus for example) prevents us from avoiding those actions in the future.  In rat simulations they artificially damaged the lateral habenula in some rats and noticed that they did not avoid actions which later had negative outcomes.

Alcohol use is interesting as 'small' use has small euphoric effects and a small (non-existent?) negative effect, so continued/occasional use (which we note as not being an alcoholic) is not an issue.  If occasional 'large' use has a large negative effect but this negative effect does not cause us to avoid these large use activities in the future then this pattern could be repeated (also necessitating larger use due to tolerances increasing) then this is what we are calling alcoholism.

It's quite interesting that even in this "modern age"(!!!) we cannot have these conversations without dragging in stigma or perceptions of mental weakness.  "Why isn't he trying harder, he's just giving up too easy, he is too weak" type observations.

Go and read the healthline article, it's really good.

On the American Overdose book thread, unfortunately it's just a continuation of stories of addiction, how lives were ruined, how bad corporations took advantage of the situation to make lots of money.  There are occasional glimpses of questions as to why only certain regions were being hit by this (which may play into the conditions paragraph as noted by the Healthline article).  But otherwise it's just a "this is terrible and unacceptable" type book.  As with people with similar attitudes (serial complainers) I often find this tiresome.  Slightly less worse are people who try to do something about it but unfortunately have not taken the time to really understand the issue and so solutions are just random and amount almost to a cargo cult type response.

Disk Performance

Recently got a Synology DS1819+ and did some performance benchmarks.

Looked around at a bunch of perf benchmark tools and settled on CrystalDiskMark 6.0.1 (although I downloaded and installed from the Microsoft Store - quick/easy install (free)).

Ran on the old Synology DS215J and on the new Synology DS1819+

Results from a CIFS mount of a folder.

DS215J:




DS1819+:



OK, so way better caching on the random reads and writes.  The sequential is about 30% better.

Disks behind the test are:

DS215J    - 2 WD30EFRX-68EUZN0 - 3TB 5400RPM WD Red disks
DS1819+ - 2 WD40EFRX-68N32N0 - 4TB 5400RPM WD Red disks


Interesting to note the WD30EFRX has 147MB/s interface speeds, the WD40EFRX has 150MB.  The 2GB model WD20EFAX has 180MB/s interface speed and 256MB cache whereas the other models have 64GB cache.  Strange difference, 2GB model seems to be much better.

There are no real specs of I/O perf of these models but the assumption is newer ones have higher areal density so better (faster) I/O.

Group Consensus

Learned from Ted Talk How can groups make good decisions? that if asked to come up with estimates, individuals on average do worse than smaller groups of 5 (estimate on height of Eiffel tower).  In moral issues small groups tended to cancel out (suppress?) more extreme views.  Presenters didn't advocate but implied that this might help out in voting situations to mediate extremism.