It’s time to replace Zsh with a saner shell because “unsetopt multifuncdef” breaks tab completion

Preface: I switched to Zsh roughly seven years ago. Prior to that I used Ksh93 for a decade. I’ve used many other UNIX shells prior to that (going back to approximately 1985 when I got my hands on my first AT&T SysV UNIX system). I’ve also used numerous shells on non-UNIX operating systems including IBM mainframes. So I like to think I’m not narrow-minded and parochial on issues such as which command shell is best.

On the zsh-users mailing list someone recently wrote about a zsh behavior that surprised them. The person ran

$ git add foo().bar

That created three functions named git, add, and foo. That’s because Zsh by default allows multiple function names when defining a function. This is considered a feature by the Zsh community. Worse, it is enabled by default and you can disable it. I view both capabilities as two of the many ill-advised features that has turned zsh into a shell whose behavior is almost impossible to understand or predict.

After reading that message thread I figured it would be a good idea to disable this feature in my interactive shells so I added

unsetopt multifuncdef

to my ~/.zshrc file. Imagine my surprise when a few days later after rebooting my computer and starting fresh shells finding that any attempt to invoke tab completion results in this error:

_main_complete:143: parse error near `()'

That’s because the /usr/share/zsh/5.0.8/functions/_main_complete file contains this block of code:

TRAPINT TRAPQUIT() {
    zle -M "Killed by signal in ${funcstack[2]} after ${SECONDS}s";
    zle -R
    return 130
}

Because zsh has no concept of modules or namespaces (other than function scope) changing an option in an interactive shell can readily break any function that is autoloaded by that shell; such as the completion functions.

Frankly, I’ve encountered too many such annoyances with Zsh. Even the developers who answer questions on the zsh-users mailing list frequently do the virtual equivalent of shrugging their shoulders and saying that some behavior or other is weird but it’s too late to change it. Not to mention too many of them seem to think it is a good thing that Zsh encourages writing code more cryptic than your typical Perl programmer would ever dream of. Such as this:

_comp_colors+=( "=(#i)${prefix[1,-2]//?/(}${prefix[1,-2]//(#m)?/${MATCH/$~toquote/\\$MATCH}|)}${prefix[-1]//(#m)$~toquote/\\$MATCH}(#b)(?|)*==$tmp" )

Or this:

list=(${${${(0)"$(git config -z --get-regexp '^alias\.')"}#alias.}%$'\n'*})

Bye-bye, Zsh. It’s time to switch to a saner shell.

P.S., Yes, I understand I could simply file a bug report to make the standard completion code robust in the face of a user unsetting that option. The point is that this is not an isolated incident. It reflects a fundamental problem with zsh trying to be all things to all people.

P.P.S., This bug apparently only existed in zsh v5.0.8 (the version that currently ships with Mac OS X 10.11 “El Capitan”). Great, someone noticed and fixed the problem quickly. That doesn’t negate my broader point that zsh simply has too many ad-hoc features that interact in surprising ways.

Updated 2015-10-28: Over the next couple of days I’m going to look closely at Xonsh and Fish for interactive use. If I don’t choose either of those I’ll probably go back to Ksh93. For scripting I’m going to switch to Bash.
Updated 2015-10-29: Two days ago a discussion was started about extending the recursive globbing syntax. Today one of the primary developers posted a patch to implement yet another configurable option to alter how recursive globbing works. With no discussion regarding alternatives, potential problems, whether the added complexity is worthwhile, etc. This is exactly the type of hastily implemented change that has made zsh a kitchen sink of features that don’t always play well together. And is another example for why I’m abandoning zsh for a more stable shell.

I would rather be unemployed than forced to write code in PHP

My blog currently uses WordPress. I’ve written numerous times about the various PHP based attacks I see every day because of the stupid security mistakes PHP programmers make. I’ve also made a few changes to the WordPress software to make it saner about handling and logging requests. Thus I knew PHP was awful from my own limited interaction with it. Then I came across this article: PHP: a fractal of bad design. This one point from that article should be enough to result in a death sentence for the language:

PHP’s one unique operator is @ (actually borrowed from DOS), which silences errors.

Holy shit! The developer(s) of PHP remind me of a coworker in my first post college job. He thought he could design and implement a new language. Yet he had no idea what the computer science terms “parser”, “lexical analysis”, “tokenizer” etc. meant. I suspect he would be welcomed by the PHP community.

Interesting new WordPress attack signature using POST /xmlrpc.php

Today I noticed an interesting, and hitherto unseen, attack from 5.152.192.218 which is owned by cloud provider redstation.com (or redstation.co.uk if you prefer). The attack started with this request:

POST /xmlrpc.php HTTP/1.0
Host: www.skepticism.us
Content-Type: application/x-www-form-urlencoded
Content-Length: 101

<?xml version="1.0"?><methodCall><methodName>demo.sayHello</methodName><params></params></methodCall>

Note the ancient HTTP/1.0 protocol specification. The methodCall is also ill-formed causing PHP to issue a notice and warning messages about Undefined index: VALUE and Invalid argument supplied for foreach().

That request was followed by another POST /xmlrpc.php that attempted to use the system.multicall method; something I’ve never seen in an attack before now. The “multicall” methods were all wp.getCategories invocations with my user ID and various passwords. In the past six months (as far as my logs go) I only started seeing attempts to exploit wp.getCategories two days ago. And this attack was the first one to do so by using system.multicall to reduce the number of requests it had to make to test which, if any, of large number of passwords was valid

A few minutes after writing the previous text I noticed that I had in fact seen another attack employing the system.multicall method to execute wp.getCategories multiples times in a single request. That attack was from ttnetdc.com in Turkey. That attack was very different. First, it was not preceded by the demo.sayHello request. Second, the wp.getCategories calls all used the generic admin account rather than my account. Third, the XML was formatted in a more or less human readable form rather than the tightly packed sequence of tokens from the attack I saw this morning and talk about above.

Thus it appears that a general approach about how to efficiently test for valid WordPress credentials was recently documented and we’re now seeing various hackers attempt to exploit that advice.

Twitter needs to hire a competent software engineer to fix their web crawler

This evening I posted an article about an Indiana State Police trooper who uses his position of power to proselytize to motorists he stops. That resulted in Twitter crawling my web server. Which would be fine but the first four requests, in a 715 ms interval, were GET /robots.txt. Every single request request came from the same address. Every single response was a HTTP 200 status that included the contents of the robots.txt file. Every single response took less than one 1 ms. What the fuck? How hard is it to avoid duplicate requests from a queue (hint: it’s pretty fucking easy)?

I went to the Twitter web page in the hope of finding an email address or web form where I could provide some constructive feedback regarding their web crawler. If it exists I couldn’t find it after searching for nearly ten minutes.

Indiana state trooper pulls woman over and asks “Do you accept Jesus Christ as your savior?”

A lot of Christians criticized me when I pointed out that WranglerStar, a YouTube content creator, should keep his religious views to himself after he spent the last minute of a six minute, ostensibly secular, video proselytizing.

While that persons behavior was inappropriate the behavior of Indiana State Police Trooper Brian Hamilton is so egregious he should be fired immediately. According to this Daily Kos story Trooper Hamilton used his position of power to proselytize for his religion after stopping a motorist for a chicken-shit offense for which he issued a verbal warning. While detaining the motorist he asked multiple questions unrelated to the traffic infraction. For example, “Did she accept Jesus Christ as her savior?” He also handed the driver a pamphlet from his preferred church. Holy shit! What motorist in the same situation would not feel intimidated to provide the answers Trooper Hamilton wanted to hear rather than tell him it’s none of his fucking business?

Fortunately the ACLU has filed a lawsuit.

Updated 2015-10-18: Sigh. This story is a year old. I really, really, hate it when an otherwise reputable site like Daily Kos doesn’t make it clear that they’re talking about ancient history.

I had to scroll to the seventh page of Google search results to find this link to PacerMonitor.com that provides some details about the lawsuit. It says the case terminated 2015-04-03 but provides no details regarding the disposition of the lawsuit.

After a lot of searching the only web page I could find that was not about the original incident and dated October 2014 was this article dated 2015-09-24. It talks about an accident to which Cpl. Brian Hamilton gave an official statement. Is that the same Brian Hamilton that was working for the Indiana State Police a year earlier? I would bet it is the same individual but the name is common enough that it could be coincidence. So, as all too often happens, it appears a “bad apple” simply moved from one police department to another.

Regular expressions: “Now you have two problems”

I’ve used the Zsh shell as my primary command line and scripting shell for the past seven years; and before that Korn shell for over a decade. Recently on the zsh-users mailing list someone asked for help that resulted in a recommendation to use a negative look-ahead regular expression.

Mikael Magnusson correctly pointed out

As a sidenote, (^foo)* is always useless to write,
since (^foo) will expand to the empty string, and then
the * will consume anything else. A useful way to think
of (^foo) is a * that will exclude any matches that
don't match the pattern foo.

To which I replied that people should Google “regular expression negative lookahead”. Which will result in numerous articles talking about Jamie Zawinski’s observation:

Some people, when confronted with a problem, think “I know, I’ll use regular expressions.” Now they have two problems.

I wholeheartedly agree with that sentiment. Notwithstanding the fact I still employ regular expressions every single day. The important thing being that I avoid them outside of ad-hoc interactive searches unless I have expended considerable thought about their correctness and failure modes if handed malformed input.

Watch Netflix’s “Black Mirror” if you like “Orphan Black”

I just binge watched the first two seasons of the British TV program Black Mirror on Netflix. If you liked Orphan Black you should definitely watch Black Mirror. Fuck that. You should watch Black Mirror even if you didn’t like Orphan Black.

Each episode of Black Mirror stands alone and is completely unrelated to the other episodes. Unrelated other than the fact that each one will make you think about and how we interact with each other and technology. Every single episode made me think about my own interactions with technology (e.g., the smart phone most of us carry with us 24/7). Not to mention my own base instincts about retributive justice or what it means to interact with someone I love.

Very few TV shows have affected me as deeply as Black Mirror. A similar TV series which was too short lived was The Booth at the End on Hulu.

Thailand has reached #1 in attacks against my server

The number of attacks from Thailand has been a significant fraction of the total for several months. In the past 24 hours I saw attacks from 51 address in Thailand, 241 in the past week. That exceeds the runner-up country (US) by a factor of five. Ten months ago I noted that Italy was the source of a disproportionate number of attacks.

Every single recent attack from Thailand has attempted to register a bogus WordPress account via a POST /wp-login.php?action=register request. Some piece of malware has managed to successfully infect a huge number of personal computers in Thailand and nowhere else. All of the computers are in the totbb.net domain

Below is the most recent such request. The details of the user login and email vary but the other details are pretty consistent.

P.S., I recognize that the numbers I’m reporting are insignificant compared to most web servers let alone the Internet as a whole. But that’s the point. My web server (blog) is only a little over a year old. My server is itself insignificant. Which means I have relatively little traffic to wade through. Which makes detecting some problems and trends easier.

POST /wp-login.php?action=register HTTP/1.1
Host: www.skepticism.us
Cookie: wordpress_test_cookie=WP+Cookie+check
Connection: Keep-Alive
User-Agent: Opera/9.80 (Windows NT 6.2; Win64; x64) Presto/2.12.388
Version/12.17
Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png,
image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Language: en
Accept-Encoding: gzip, deflate
Referer: http://www.skepticism.us/wp-login.php?action=register
Content-Type: application/x-www-form-urlencoded
Content-Length: 109

user_login=PattiThorne3&user_email=pattisabj9571%40admin2%40metalchopsaw.info&redirect_to=&wp-submit=Register

VPS/cloud provider soladrive.com has no working abuse or support email

Today my server was attacked by a server owned by soladrive.com; specifically IP addr 96.44.156.75 (which is also owned by quadranet.com). I sent a report of the attack to abuse@soladrive.com. That email was rejected with this text:

  pipe to |/home/sola/public_html/support/pipe/pipe.php
    generated by abuse@soladrive.com
    local delivery failed

I then forwarded that to support@soladrive.com — the address I found on their web page. That also bounced with the same error. WTF! Why would anyone use a VPS provider without a working abuse email address? Oh, right. They use companies like this because they don’t give a shit about security or abuse from their network.

Shortly after writing the above text I sent an email to sales@soladrive.com which is also on their public web page as a point of contact. It too bounced with the same error. Jebus H Christus. The people running Soladrive.com appear to be morons.

An alcoholic on the bus ride home tonight… WTF is wrong with our mental health support network

I rode the bus this afternoon to assist the San Jose Downtown Streets Team in putting homeless people back to work. On the bus ride back home I noticed an individual on the bus I initially thought was blind and/or mentally handicapped. But a few minutes after I boarded he started shouting seemingly random phrases. He then made several random seating changes (while showing us his lovely butt crack).

Soon after that he slowly took off his pants. While doing so the bus driver calmly told him to put his pants back on. The rider then stood up (completely naked from the waste down) and slowly put on shorts. The bus driver pulled over at the next stop and could be seen talking to his dispatcher. The rider then moved to the very back of the bus where I observed him drinking what appeared to be straight vodka while intermittently spitting and drooling on the floor. I suspect the alcoholic had pissed himself thus prompting the change of pants.

The bus driver was very polite to the alcoholic and told him the sheriff would be arriving soon and he could wait on the bus or on the sidewalk. The alcoholic chose to wait on the bus. At which point the driver suggested to the rest of us that we should consider changing to the next route #22 bus that was about to arrive at our location.

The bus driver should be commended for an excellent job in handling a difficult and uncomfortable situation. I wish I had stayed to observe how the police handled the situation. I hope the police exhibited the same professionalism and compassion shown by the bus driver but I would be surprised if they did so. Hopefully this person will get some help for his addiction but I doubt that will happen given that our state run mental health programs have been underfunded for decades.