Optimizing Objects in JavaScript

Posted by on 2020/12/21 No comments

I have web applications with JavaScript business objects coming from constructors, like:

function Animal() {
  this.type      = null;   // 'Cat'
  this.color     = null;   // 'Black'
  this.legs      = 0;      // 4
  this.carnivore = false   // true
}

These objects may be created on the web, but quite quickly this happens:

  1. Client
    1. creates an Object
    2. serializes Object with JSON.stringify()
  2. The JSON text is sent over the network from client to the server
  3. Server
    1. uses JSON.parse() to get the object back
    2. validates the object
    3. stores it (perhaps by doing JSON.stringify())
  4. Client (perhaps the same) asks the server which sends it over the network
  5. Client
    1. uses JSON.parse to get the object back
    2. keeps the object in memory for application logic use

This works fine! If you do not add prototypes/functions to your constructors (like Animal.prototype.eat()) the client can use the object it got from JSON.

Since I send millions of such objects in this way over the network every day, I can’t help asking myself if what I do is reasonably efficient, or rather wasteful?

One idea I have had is that for this purpose I could turn Objects into Arrays (when shipping over the network, or perhaps storing them), like:

// As Object                          // As Array
{                                     [
  type      : 'Cat',                    'Cat',
  color     : 'Black',                  'Black',
  legs      : 4,                         4,
  carnivore : true                       true
}                                     ]

Another idea has been to create an object with “new Animal()” rather than using the raw Object I get from JSON.parse().

Possible benefits could be

  1. Arrays are smaller to store on disk
  2. Arrays are smaller to send over the network
  3. A real Animal Object may be stored more efficiently in client RAM than the raw Object
  4. A real Animal Object may be faster to operate on, in the client, than the raw Object

So rather than just sending, receiving and processing raw JSON, I could be sending and receiving Arrays, and create objects using their constructors.

Test Results

I implemented some Node.js code to test my ideas. I was using objects like:

// As Object                                 // As Array
{                                            [
  "theWorst":0.1560387568813406,               0.1560387568813406,
  "lowerQuartile":0.2984895507275531,          0.2984895507275531,
  "median":0.47865973555734964,                0.47865973555734964,
  "higherQuartile":0.7832137265963346,         0.7832137265963346,
  "theBest":0.8893834668143412                 0.8893834668143412
}                                            ]

When there is a memory range (below), the low value is after the GC has run, and the high value is the peak value. JSON means an object received from JSON.parse. Object means an Object created with a constructor.

Intel i7-8809GRAM/DiskCPU
1M Arrays94MB
-> gzip43MB7.7s
-> gunzip1.1s
1M Objects154MB
-> gzip48MB5.6s
-> gunzip1.3s
Receive & Convert data
Arrays->Arrays100-240MB0ms
Arrays->Objects76-240MB334ms
JSONs->JSONs123-310MB0ms
JSONs->Objects76-382MB280ms
Access & Use data
Arrays21ms
JSONs25ms
Objects9-11ms

Well, I find that:

  1. It is surprising that GZIP is more expensive on the smaller array than the larger object file.
  2. Costs (CPU) to compress/decompress much higher (~10x) than the cost of “packing/unpacking” JSON-data in JavaScript code.
  3. If we are using gzip for network traffic the benefit of sending the more compact arrays rather than the more wordy objects, is questionanable (higher CPU cost, 10% smaller end result).
  4. Arrays like this require basically the same amount of RAM in Node.js as disk space.
  5. Objects like this require less RAM in Node.js than the corresponding JSON file.
  6. Both when it comes to RAM usage and performance on the client side, Arrays are better than raw JSON objects, but worse than real objects from a Constructor.
  7. Unless an object is used many times on the client (10+) it is not worth it from a strict performance perspective to make it with its constructor, instead of the raw JSON.

When it comes to the different strategies I thus find:

IO/Stored formatJavaScript formatConclusion
ArrayArrayanimal[1] or animal[COLOR] (where COLOR is a global constant) is generally not acceptable compared to animal.color. And it is not justified from performance perspective either.
ArrayJSONThis would not happen
ArrayObjectGiven the extra cost of gzip, and the significant complexity of serializing/deserializing, this is hardly a good general strategy. It requires the least disk space, the least network traffic, and the least RAM on the client though.
JSONArrayThis would not happen
JSONJSONThis is the most simple, readable way of doing things, at a surprisingly low overhead. You can not use prototype though.
JSONObjectIf you find a simple and reliable way to create objects with a construtor and populate it from a JSON object, this method is probably best for performance and efficiency.

Conclusion

JSON is very simple and using it thoughout your full stack is very productive. Unless you really need to optimize things, write your code for pure raw JSON objects.

Great JavaScript Stuff 2020

Posted by on 2020/11/28 No comments

I never write about great new JavaScript features on this blog. That is because there is really nothing you can not do without great new features, and I prefer writing compatible code (that runs in Internet Explorer).

However some code is backend-only, and if Node.js supports a feature I can use it.

So, here are two great new things in JavaScript that I will start using in Node.js.

Optional Chaining

It is very common in JavaScript with code like:

price = inventory.articles.apple.price;

If inventory, articles or apple is null or undefined this will throw an error. One common way to do this is:

price = inventory &&
        inventory.articles &&
        inventory.articles.apple &&
        inventory.articles.apple.price;

That is obviously not optimal. I myself have implemented a little function, so I do:

price = safeget(inventory,'articles','apple','price');

The elegant 2020 solution is:

price = inventory?.articles?.apple?.price;

Who wouldn’t want to do that? Well, you need to be aware that it will “fail silently” if something is missing, so you should probably not use it when you dont expect anything to be null or undefined, and not without handling the “error” properly. If you replace . with ?. everywhere in your code, you will just detect your errors later when your code gives the wrong result rather than crashing.

Nullish Coalescing

It is very common to write code like:

connection.port = port || 80;
connection.host = server || 'www.bing.com';
array = new Array(size || default_size);

The problem is that sometimes a “falsy value” (0, false, empty string) is a valid value, but it will be replaced by the default value. The port above can not be set to 0, and the host can not be set to ”;

I often do things like:

connection.port = 'number' === typeof port ? port : 80;
connection.host = null == server ? 'www.bing.com' : server;

However, there is now a better way in JavaScript:

connection.port = port ?? 80;
connection.host = server ?? 'www.bing.com';
array = new Array(size ?? default_size);

This will only fall back to the default value if port/server/size is null/undefined. Much of the time, this is what you want.

However, you still may need to do proper validation, so if you used to do:

connection.port = validatePort(port) ? port : 80;

you shall probably keep doing it.

Conclusion

If your target environment supports Optional chaining and Nullish coalescing, take advantage of it. Node.js 14 supports it.

Functional Programming is Slow – revisited

Posted by on 2020/11/07 2 comments

I have written before about Functional Program with a rather negative stand point (it sucks, it is slow). Those posts have some readers, but they are a few years old, and I wanted to do some new benchmarks.

Please note:

  • This is written with JavaScript (and Node.js) in mind. I don’t know if these findings apply to other programming languages.
  • Performance is important, but it is far from everything. Apart from performance, there are both good and bad aspects of Functional Programming.

Basic Chaining

One of the most common ways to use functional programming (style) in JavaScript is chaining. It can look like this:

v = a.map(map_func).filter(filter_func).reduce(reduce_func)

In this case a is an array, and three functions are sequentially called to each element (except reduce is not called on those that filter gets rid of). The return value of reduce (typically a single value) is stored in v.

  • What is the cost of this?
  • What are the alternatives?

I decided to calculate the value of pi by

  1. evenly distribute points in the[0,0][1,1] rectangle.
  2. for each point calculate the (squared) distance to origo (a simple map)
  3. get rid of each point beyond distance 1.0 (a simple filter)
  4. count the number of remaing points (a simple reduce – although in this simple case it would be enough to check the length of the array)

The map, filter and reduce functions looks like:

const pi_map_f = (xy) => {
  return xy.x * xy.x + xy.y * xy.y;
};
const pi_filter_f = (xxyy) => {
  return xxyy <= 1.0;
};
const pi_reduce_f = (acc /* ,xxyy */) => {
  return 1 + acc;
};

In chained functional code this looks like:

const pi_higherorder = (pts) => {
  return 4.0
       * pts.map(pi_map_f)
            .filter(pi_filter_f)
            .reduce(pi_reduce_f,0)
       / pts.length;
};

I could use the same three functions in a regular loop:

const pi_funcs = (pts) => {
  let i,v;
  let inside = 0;
  for ( i=0 ; i<pts.length ; i++ ) {
    v = pi_map_f(pts[i]);
    if ( pi_filter_f(v) ) inside = pi_reduce_f(inside,v);
  }
  return 4.0 * inside / pts.length;
};

I could also write everything in a single loop and function:

const pi_iterate = (pts) => {
  let i,p;
  let inside = 0;
  for ( i=0 ; i<pts.length ; i++ ) {
    p = pts[i];
    if ( p.x * p.x + p.y*p.y <= 1.0 ) inside++;
  }
  return 4.0 * inside / pts.length;
};

What about performance? Here are some results from a Celeron J1900 CPU and Node.js 14.15.0:

Iterate (ms)Funcs (ms)Higher Order (ms)Pi
10k88103.1428
40k34193.1419
160k33473.141575
360k661963.141711
640k11114043.141625
1000k17175593.141676
1440k252511603.14160278

There are some obvious observations to make:

  • Adding more points does not necessarily give a better result (160k seems to be best, so far)
  • All these are run in a single program, waiting 250ms between each test (to let GC and optimizer run). Obvously it took until after 10k for the Node.js optimizer to get things quite optimal (40k is faster than 10k).
  • The cost of writing and calling named functions is zero. Iterate and Funcs are practially identical.
  • The cost of chaining (making arrays to use once only) is significant.

Obviously, if this has any practical significance depends on how large arrays you are looping over, and how often you do it. But lets assume 100k is a practical size for your program (that is for example 100 events per day for three years). We are then talking about wasting 20-30ms every time we do a common map-filter-reduce-style loop. Is that much?

  • If it happens server side or client side, in a way that it affects user latency or UI refresh time, it is significant (especially since this loop is perhaps not the only thing you do)
  • If it happens server side, and often, this chaining choice will start eating up significant part of your server side CPU time

You may have a faster CPU or a smaller problem. But the key point here is that you choose to waste significant amount of CPU cycles because you choose to write pi_higherorder rather than pi_funcs.

Different Node Versions

Here is the same thing, executed with different versions of node.

1000kIterate (ms)Funcs (ms)Higher Order (ms)
8.17.01111635
10.23.01111612
12.19.01111805
14.15.01819583
15.1.01719556

A few findings and comments on this:

  • Different node version show rather different performance
  • Although these results are stable on my machine, what you see here may not be valid for a different CPU or a different problem size (for 1440k points, node version 8 is the fastest).
  • I have noted before, that functional code gets faster, iterative code slower, with newer versions of node.

Conclusion

My conclusions are quite consistent with what I have found before.

  • Writing small, NAMED, testable, reusable, pure functions is good programming, and good functional programming. As you can see above, the overhead of using a function in Node.js is practially zero.
  • Chaining – or other functional programming practices, that are heavy on the memory/garbage collection – is expensive
  • Higher order functions (map, filter, reduce, and so on) are great when
    1. you have a named, testable, reusable function
    2. you actually need the result, just not for using once and throwing away
  • Anonymous functions fed directly into higher order functions have no advantages whatsoever (read here)
  • The code using higher order functions is often harder to
    1. debug, becuase you can’t just put debug outputs in the middle of it
    2. refactor, because you cant just insert code in the middle
    3. use for more complex agorithms, becuase you are stuck with the primitive higher order functions, and sometimes they don’t easily allow you to do what you need

Feature Wish

JavaScript is hardly an optimal programming language for functional programming. One thing I miss is truly pure functions (functions with no side effects – especially no mutation of input data).

I have often seen people change input data in a map.

I believe (not being a JavaScript engine expert) that if Node.js knew that the functions passed to map, filter and reduce above were truly pure, it would allow for crazy optimizations, and the Higher Order scenario could be made as fast as the other ones. However, as it is now, Node.js can not get rid of the temporary arrays (created by map and filter), because of possible side effects (not present in my code).

I tried to write what Node.js could make of the code, if it knew it was pure:

const pi_allinone_f = (acc,xy) => {
  return acc + ( ( xy.x * xy.x + xy.y * xy.y <= 1.0 ) ? 1 : 0);
};

const pi_allinone = (pts) => {
  return 4.0
       * pts.reduce(pi_allinone_f,0)
       / pts.length;
};

However, this code is still 4-5 times slower than the regular loop.

All the code

Here is all the code, if you want to run it yourself.

const points = (n) => {
  const ret = [];
  const start = 0.5 / n;
  const step = 1.0 / n;
  let x, y;
  for ( x=start ; x<1.0 ; x+=step ) {
    for ( y=start ; y<1.0 ; y+=step ) {
      ret.push({ x:x, y:y });
    }
  }
  return ret;
};

const pi_map_f = (xy) => {
  return xy.x * xy.x + xy.y * xy.y;
};
const pi_filter_f = (xxyy) => {
  return xxyy <= 1.0;
};
const pi_reduce_f = (acc /* ,xxyy */) => {
  return 1 + acc;
};
const pi_allinone_f = (acc,xy) => {
  return acc + ( ( xy.x * xy.x + xy.y * xy.y <= 1.0 ) ? 1 : 0);
};

const pi_iterate = (pts) => {
  let i,p;
  let inside = 0;
  for ( i=0 ; i<pts.length ; i++ ) {
    p = pts[i];
    if ( p.x * p.x + p.y*p.y <= 1.0 ) inside++;
  }
  return 4.0 * inside / pts.length;
};

const pi_funcs = (pts) => {
  let i,v;
  let inside = 0;
  for ( i=0 ; i<pts.length ; i++ ) {
    v = pi_map_f(pts[i]);
    if ( pi_filter_f(v) ) inside = pi_reduce_f(inside,v);
  }
  return 4.0 * inside / pts.length;
};

const pi_allinone = (pts) => {
  return 4.0
       * pts.reduce(pi_allinone_f,0)
       / pts.length;
};

const pi_higherorder = (pts) => {
  return 4.0
       * pts.map(pi_map_f).filter(pi_filter_f).reduce(pi_reduce_f,0)
       / pts.length;
};

const pad = (s) => {
  let r = '' + s;
  while ( r.length < 14 ) r = ' ' + r;
  return r;
}

const funcs = {
  higherorder : pi_higherorder,
  allinone : pi_allinone,
  functions : pi_funcs,
  iterate : pi_iterate
};

const test = (pts,func) => {
  const start = Date.now();
  const pi = funcsfunc;
  const ms = Date.now() - start;
  console.log(pad(func) + pad(pts.length) + pad(ms) + 'ms ' + pi);
};

const test_r = (pts,fs,done) => {
  if ( 0 === fs.length ) return done();
  setTimeout(() => 
    test(pts,fs.shift());
    test_r(pts,fs,done);
  }, 1000);
};

const tests = (ns,done) => {
  if ( 0 === ns.length ) return done();
  const fs = Object.keys(funcs);
  const pts = points(ns.shift());
  test_r(pts,fs,() => {
    tests(ns,done);
  });
};

const main = (args) => {
  tests(args,() => {
    console.log('done');
  });
};

main([10,100,200,400,600,800,1000,1200]);

Roleplaying in Middle Earth & magic

Posted by on 2020/10/18 No comments

Middle Earth is a fantastic fantasy world and it can be used for tabletop roleplaying games. There are games designed with Middle Earth in mind (MERP by Iron Crown, The One Ring by C7, Adventures in Middle Earth by C7), but they are all out of print. Given the vast amount of Middle Earth content available elsewhere it should be quite fine to use any fantasy roleplaying system not written for a specific setting, right? I particularly like the simplicity of OSR. Well, it is that thing with magic.

Magic in fantasy roleplaying games

Magic in fantasy RPGs is most often about spells. Sometimes they are spectacular and action oriented. Often there are clerics getting their powers from deities, and sorcerors taming raw magic power.

This is not obviously a good match for Middle Earth. Clerics praying to Valar for spells is kind of unheard of. Saruman and Gandalf use magic, but not so easily and not very spectacularly, and they are Maia (half god-like spirits) not men. Powerful elves have some magic abilities. But regular men being educated wizards casting fireballs is quite not in the books.

Magic in Middle Earth

I am presuming a (late) Third Age campaign.

There is definitely magic in Middle Earth. Many magical items are mentioned. There are things like the river horses that swept away the nazgul outside Rivendell, the tempest Saruman used to stop the fellowship from crossing the misty mountains, and whatever was done to Theoden.

However, most magic is very old, connected directly with Elves, or connected to Maia.

When Sauron appeared in Dol Guldur he was thought to be a Necromancer. So the free people of Middle Earth found the existence of a necromancer a possibility (more likeely than the return of Sauron). If a powerful necromancer could have appeared there should also exist magic practitioners.

The question is, how and to what extent can magic be available to the player characters, and their enemies?

What are we looking to solve?

Obviously, if GM and players agree, then so is it: Clerics worhipping Valar and fireballs. However, I think we can do better.

Also, if we want much magic (like in Forgotten Realms) then perhaps it is better to play in setting with more open magic (like FR). But I think we can do better.

Sam feared that Gandalf would turn him into something unnatural. In the movie Gimli warns the hobbits of the elf witch who lives in Lorien […] all under her spell and are never seen again. No one in the fellowship (except Gandalf) had imagined the Balrog in their darkest dreams. So I argue that it is in the essential nature of Middle Earth that unexpected magic and supernatural things can happen. If you mostly remove magic you actually turn your Middle Earth game into less than it deserves to be.

I would like to have a Middle Earth where magic is not at all commonplace, but where it can feasibly and arguably exist. Ideally, I would like to interpret and explain magic so that I can mostly use simle OSR rules, with moderation.

Ideas and possibilities

I will now present some arguments for why (OSR style) magic could be used to some degree in a late Third Age Middle Earth RPG campaign.

Alatar and Pallando: Tolkien wrote in a letter about the two blue Istari: I suspect they were founders or beginners of secret cults and ‘magic’ traditions that outlasted the fall of Sauron. This is clearly an opening for the existence of sorcerors or wizards in Middle Earth, even though they played no part in the books.

Religion: We don’t read in the books about clerics praying and sacrificing to Valar. But the Third age is 3000 years old, and it is not hard to imagine that the free peoples were actually worshipping and praying. That works of fiction do not mention religion does not mean religion can not exist. The most important building in Gondor was obviously not a cathedral, that we know. But can there be no shrines, nowhere, at all?

However, worshipping is one thing, being granted spells is a different story.

Valar: Since the end of Second Age, Valar has a kind of non-intervention policy. However, they sent the Istari to Middle Earth on a mission. Someone sent Gandalf back as the white wizard after the fight with the Balrog. So I think it is unnecessarily restrictive to imagine they have no power whatsoever in Middle Earth.

Maia: The Maia helped Valar create the world, so they are powerful magical spirits creating and changing reality. All of them are not named and they are not numbered. Gandalf said: There are older and fouler things than Orcs in the deep places of the world. The Balrog had been hidden for long times when the dwarves disturbed it. I think as a GM you have quite some space to use unmentioned Maia as source of magic powers.

End of Third Age: The Valar has been quite absent for three millennia. Sauron is rising but driven from Dol Guldur, Smaug is killed, and large parts of Middle Earth (Arnor) is largely desolate and open. The Istari is not supposed to use force, but Saruman has other plans. This is a situation with two factors:

  • it is a power vacuum – there for somone (Maia or other) to fill
  • the rules are rewritten – Istari are failing their mission

And things can have been going on for centuries quite secret in the background.

Multiverse: The first chapter of Silmarillion does not start with These events took place in a little corner of the multiverse. No other creation myths start like that either. Nevertheless the multiverse idea is that different game settings exist in simultaneously and that the deities can be present in several of them. If Middle Earth is really left alone by the Valar, why would not some other power show interest in it?

Leave it open

I am not suggesting that you as a GM write down exactly who is granting or teaching what magic powers in Middle Earth and tell your players all about it. That serves no purpose and the players need not know.

What you tell the players is that just as the fellowship did not know about the Balrog, they also don’t know about all powers of Middle Earth.

You can tell that the Istari – or other unknown Maia – during the last millennia of the third age may have taught, shared or granted magic to smaller groups of rather unknown spellcasters. The characters know nothing of this, of course, but the players may ask.

You are now free, with moderation, to introduce spellcasters as part of your adventures.

And if a players wants to be a cleric, wizard, sorceror or warlock, it can be done. It may come with risks though

  • being accused of witchcraft and black magic is always dangerous
  • that magic teacher or spell-granting-deity, who is it really?

Sauron decieved the Numenoreans to worship Morgoth and make human sacrifices.

Conclusion

I think it is possible to run a Middle Earth campaign with a simple OSR system (or your favorite game) and use the magic system of that game mostly the way it is. That does not mean that magic appears in Middle Earth just as it does in Forgotten Realms (or some other hi-magic setting). It just means that magic is possible.

I don’t suggest you to turn Middle Earth into a hi-magic setting. But I also think it is a mistake to have so little magic that there is nothing mysterious or unexpected to the players and their characters to discover.

D&D: OSR or 5th edition?

Posted by on 2020/10/10 No comments

There are many versions of D&D and there are things to be aware of before you just go with the latest and very popular 5th edition.

All versions of D&D and the so called Old-School-Revival retroclones share many things. Characters have hit dice depending on class (1d4 to 1d12 per level), with slightly more generous dice in newer versions.

Monsters also have hit dice (usually number of d8 HP). Look at this little table of average HP for different monsters in different versions:

AD&D 1eAD&D 2eD&D 3eD&D 5e
Men/Bandit1-6 (4)2d8+2 (11)
Bugbear3+1 (15)3+1 (15)3d8+3 (17)5d8+5 (27)
Hydra (5H)5 (23)5 (23)5d10+28 (55)15d12+75 (172)
Owlbear5+2 (25)5+2 (25)5d10+25 (52)7d10+21 (59)
Werewolf4+3 (21)4+3 (21)3d8+7 (21)9d8+18 (58)
Zombie2 (9)2 (9)2d12+3 (16)3d8+9 (22)

Weapons make roughly the same damage (Longsword 1d8) in all versions.

About Men/Bandit, Monster Manual for 1e states: For every 20 bandits encountered there will be an additional 3rd level fighter. Compare this to 5e, where every bandit essentially is level 2.

Consequences for fights

Fights in 1e/2e are faster because enemies have less hitpoints. They are also less detailed than 3e/5e where special abilities, feats, action economy, bonus actions and reactions are part of every round. In 1e/2e it is more like you have one attack and you can move.

You can argue however you want: rolling/counting to 21 (werewolf 2e) is quicker than rolling/counting to 58 (5e).

Sometimes this difference is called “fighting as battle vs fighting as sport”.

5e is more demanding on the DM and the Players when it comes to tactics. It is more like a miniature war game or collectible card game where it pays of for the players to create combos, optimize characters and prepare tactics for their group or characters.

Consequences for the World

AD&D 1e/2e comes with the idea that most people in the world are common people. They have no class and no special abilities. This applies to bandits, soldiers but also to orcs and goblins that the PCs may encounter. Veteran was the title of a 1st level fighter in Basic D&D.

D&D 5e comes with the idea that common enemies are much more than ordinary people. Veteran in 5e monster manual has 9d8+18 HP.

Consequences for your Campaign

Don’t be mistaken and believe that 5e is deadlier and more difficult. HP is not everything, 5e is balanced differently when it comes to AC, saving throws and other things. Many people witness that the characters become super heroes quite quickly in 5e.

5e allows you to throw in more powerful goblins to challenge your more powerful characters. AD&D expects you to challenge the players with genuinely more powerful monsters, or more goblins using other tactics.

To me, AD&D feels like the PCs are becoming heroes in the same world as they started. But 5e feels like a game where the PCs travel to new places where everything is much more powerful and dangerous than where they came from (like in World of Warcraft, with different regions for players of different power).

OSR – Old School Revival

I would argue that this difference between AD&D on one hand, and 3e/5e on the other hand is significant. You can not quite play the same way in AD&D and 5e, and if you try you may end up disappointed.

After many years of not playing D&D at all, I just bought D&D 5e without thinking too much about the differences. I expected to use it like AD&D 2e. I think that was a mistake, I am not happy with 5e, and I am going back.

I am not alone. Many people prefer older versions of D&D and old style of play (Old School Revival). You and your group can make your choice!

When it comes to OSR there are many options to AD&D 1e/2e! Basic D&D (or Classic) was a simplified version of AD&D 1e (that is a simplification) that for the purposes of this article is similar to AD&D 1e/2e and different from 3e/5e. There are also retroclones that are newly edited and highly compatible inoffical versions of D&D.

AD&D 1e: Can be bought printed or as PDF from dmsguild.com. I am too young to ever have played it, I bought it for fun recently, and I don’t quite feel like playing it. It is complex in unnecessary ways.

AD&D 2e: Can be bought printed or as PDF from dmsguild.com. If you like a more advanced game, this may be your best OSR option.

Basic D&D: Can be bought as PDF from dmsguild. There are different versions (Holmes, Moldvay B/X, Mentzer BECMI) and the last version was packaged in a single book called Rules Cyclopedia that can be bought as PDF or print from dmsguild.

Basic Fantasy RPG: An entirely free to download clone.

Dungeon Crawl Classics: Based on Basic D&D, it adds a lot of innovation and it is more of a different game. Perhaps not for beginners.

Old School Essentials: A premium B/X clone. You get more content for less money if you go for Rules Cyclopedia. OSE is a arguably nicer if you prefer less is more, it has a better look and feel, and it is probably the best beginners game.

Conclusion

Play whatever version of D&D with your friends that you like! But do not just get D&D 5e assuming it is just the latest, most revised and best version. It is a rather different game from the old games of the 70s-80s-90s that made D&D popular. The differences may be subtle at first, but will become more important over time.

Testing Business Central with Docker

Posted by on 2020/09/19 No comments

There are many articles and sources on the internet about Business Central in Docker. Most of them are very specific about some detail. With this post I hope to share some ideas about why we run BC in docker, and the challenges from a top-down perspective.

When you set up any non-trivial system, automated testing is helpful. It can go something like:

  1. Write new tests
  2. Update system (code or configuration)
  3. Start system (initiate from scratch)
  4. Run tests
  5. Stop system (discard everything)
  6. Repeat

The key here is repeatability. So you want to know that the system starts to identical state every time, so the tests work every time and you know exactly what you are testing.

This used to be very hard with complex systems like Business Central (NAV). It is still not very easy, but with Business Central being available as Docker images, automated tests are viable.

Assets

I think it is important to understand exactly what defines the running system. In my Business Central tests, those essential assets are:

  • A docker image (mcr.microsoft.com/dynamicsnav:10.0.19041.508-generic)
  • An artifact (https://bcartifacts.azureedge.net/sandbox/16.5.15897.16650/se)
  • Parameters for docker run (to create container from image)
  • A Business Central license file
  • A custom script (AdditionalSetup.ps1)
  • Several Business Central Extensions
  • A rapid start configuration package

Other non-BC assets could be

  • Version of Windows and Docker
  • Code for automating 3-5 (start-test-stop) above
  • Test code

Sharing those assets with my colleagues, we shall be able to set up identical Business Central systems and run the same tests with the same result. Any upgrade of an asset may break something or everything and that can be reproduced. Also the fix can be reproduced.

Business Central in Docker

Business Central is a rather large and complex beast to run in Docker. It is not just start and stop. And you will run into complications. Primary resources are:

  • Freddys blog (you will end up there when using Google anyway)
  • NAV Container Helper (a set of PS-scripts, even reading the source code has helped me)
  • Official Documentation: APIs, Automation APIs, Powershell Tools

This is still far from easy. You need to design how you automate everything. My entire start-to-stop-cycle looks something like:

  1. I download image
  2. I run image (with parameters) to create container
  3. I start container (happens automatically after 2)
  4. Artifact is being downloaded (unless cached from before)
  5. Initial container setup is being done (user+pass created)
  6. Business Central is starting up
  7. AdditionalSetup.ps1 is run (my opportunity to run custom PS code in container)
  8. I install extensions
  9. I add (Damage Inc) and delete (CRONUS) companies
  10. I install rapid start package
  11. I run read-only-tests
  12. I run read-write-tests
  13. I stop container
  14. I remove container

There are a few things to note.

  • 1 and 3 only if not already downloaded
  • 4,5,6,7 is happening automatically inside BC docker, all I can do is observe the result (like keep user+pass)
  • It is possible to run 3-13 (when using same image and artifact, and as long as container works and gives expected results) only
  • It is possible to run 8-12 (on already running container)
  • It is possible to run 11 only (on already running container)
  • 8/9 should probably switch order in the future

Tooling

In order to automate, and automate tests, you need some tool. It can be just a scripting language or something more complicated. You need to pick tools for:

  • Starting, testing, stopping the whole thing
  • Step 8-10 can be done using Powershell (invoked step 7) or using Microsoft Automation API (so you need a tool to make http-requests)
  • Step 11-12 is about testing the BC APIs, using http-requests, so you need a tool that can handle that

I already have other systems that are being tested in a similar way, so for me Business Central is just a part of a bigger integration test process. I am using Node.js and Mocha since before, so I use it for most everything above. However, some things need to be done in Powershell (AdditionalSetup.ps1) as well, more on that later.

System Requirements

You need a reasonably good Windows 10 computer. 16GB or RAM is acceptable so far, but if you have other heavy things running or perhaps later when you get more data in BC you will find that 16GB is too little. I am doing quite fine with my 8th gen i7 CPU.

The number 19041.508 in the docker image name corresponds to my Windows version. You may not find images for some older version of Windows 10.

You are probably fine with a recent Windows Server. I have not tried.

Bascially, Windows docker images can only run on Windows computers, so Linux and Mac OS will not just work (there may be ways with virtualization, Wine or something, I dont know).

Performance

Ideally, when you do automated testing you want to be able to iterate fast. I have found that two steps take particularly long time (~10 min each).

  1. Downloading image and artifact
  2. Importing the Rapid Start Configuration Package (your company data)

Fortunately, #1 is only done the first time (or when you upgrade version).

Unfortunately, #2 is something I would like to do every time (so my tests can update data, but always run on the same data set).

Given the unfortunate #2 it does not make so much sense to put much effort into reusing the container (docker start container, instead of docker run image). I think eventually I will attempt to write read-write-tests that clean up after themselves, or perhaps divide the rapid package into several packages so I can only a last small one every test run. This is not optimal, but it is about optimization.

Nav Container Helper

Freddy (and friends) have written Nav Container Helper. You should probably use it. Since I am a bit backwards, Nav Container Helper is not part of my automated test run. But I use it to learn.

I can invoke Nav Container Helper with version and country arguments to learn what image and artifact to use.

Unfortunately documentation of BC in docker itself is quite thin. I have needed to read the source of Nav Container Helper, and run Nav Container Helper, to understand what options are available when creating a container.

Nav Container Helper annoys me. It kind of prefers to be installed and run as administrator. It can update the hosts-file when it creates a container, but that is optional. However, when removing a container it is not optional to check the hosts file, so I need to remove containers as administrator. I am also not very used to PowerShell, admittedly.

Nav Container Helper will eventually be replaced by the newer BC Container Helper.

Image and Artifact

The images are managed by docker. The artifacts are downloaded the first time you need them and stored in c:\bcartifacts.cache. You can change that folder to anything you like (see below). The image is capable of downloading the artifacts itself (to the cache folder you assign), so you don’t need NavContainerHelper for this.

To find the best generic image for you computer:

Get-BestGenericImageName

To find artifact URLs for BC, run in powershell (you need to install Nav-ContainerHelper first):

Get-BCArtifactUrl -version 16.4

Docker option and environment

When you run a docker image, which creates and starts a container, you can give options and parameters. When you later start an already existing container, it will use the same options as when created.

Since I don’t use NavContainerHelper to run image, here are options (arguments to docker run) I have found useful.

  -e accept_eula=Y
  -e accept_outdated=Y
  -e usessl=N
  -e enableApiServices=Y
  -e multitenant=Y
  -m 6G
  -e artifactUrl=https://bcartifacts.azureedge.net/sandbox/16.5.15897.16650/se
  -e licenseFile=c:\run\my\license.flf
  --volume myData:c:\Run\my
  --volume cacheData:c:\dl
  -p 8882:80
  -p 7048:7048

I will not get into too many details but:

  • You just need to accept EULA
  • The image may be old (whatever that is), use it anyway
  • I don’t care about SSL when testing things locally
  • You need to enable API to use it (port 7048)
  • Since 16.4 multitentant is required to be able to create or remove companies (you usually need to add ?tenant=default to all URLs)
  • 4GB is kind of recommended, I use 6GB now when importing rapid start packages of significant size
  • For doing anything real you most likely will need a valid license file. The path given is in the container (not on your host)
  • I have a folder (replace myData with your absolute path) on the host computer with a license file, my AdditionalSetup.ps1, and possibly more data. –volume makes that folder available (rw) as c:\run\my inside the docker container.
  • I have a folder (replace cacheData with your absolute path) where artifacts are downloaded. This way they are saved for next container.
  • Business Central UI listens to http:80. I expose that on my host on 8882.
  • Business Central API services are available on http:7048. I expose that on my host on 7048.

NavContainerHelper will do some of these things automatically and allow you to control other things with parameters. You can run docker inspect on a container to see how it was actually created.

Username & Password

First time you run a container (that is, when you create it using docker run) it will output username, password and other connection information to stdout. You may want to collect and save this information so you can connect to BC. There are ways to set a password too – I am fine with a generated one.

AdditionalSetup.ps1

If there is a file c:\run\my\AdditionalSetup.ps1 in the container, it will be run (last). You can do nothing, or a lot with this. It turned out that installing extensions via the API requires something to be installed first. So right now I have this in my AdditionalSetup.ps1:

Write-Host 'Starting AdditionalSetup.ps1'
if ( -not (Get-Module -ListAvailable -Name ALOps.ExternalDeployer) ) {
  Write-Host 'Starting ALOps installation'
  Write-Host 'ALOps 1/5: Set Provider'
  Install-PackageProvider -Name NuGet -Force
  Write-Host 'ALOps 2/5: Install from Internet'
  install-module ALOps.ExternalDeployer -Force
  Write-Host 'ALOps 3/5 import'
  import-module -Name ALOps.ExternalDeployer
  Write-Host 'ALOps 4/5 install'
  Install-ALOpsExternalDeployer
  Write-Host 'ALOps 5/5 create deployer'
  New-ALOpsExternalDeployer -ServerInstance BC
  Write-Host 'ALOps Complete'
} else {
  Write-Host 'ALOps Already installed'
}

This is horrible, because it downloads something from the internet every time I create a new container, and it occationally fails. I tried to download this module in advance and just install/import it. That did not work (there is something about this NuGet-provider that requires extra magic offline). The Microsoft ecosystem is still painfully imature.

To try things out in the container, you can get a powershell shell inside the container:

docker exec -ti <containername> powershell

Install Extensions

A usually install extensions with the cmdlets:

  1. Publish-NAVApp
  2. Sync-NAVApp
  3. Install-NAVApp

in AdditionalSetup.ps1 (before setting up companies – that seems to not matter so much). You need to “import” those cmdlets before using them:

import-module 'c:\Program Files\Microsoft Dynamics NAV\160\Service\Microsoft.Dynamics.Nav.Apps.Management.psd1'

I can also use the automation API, if I first install ALOps.ExternalDeployer as above (but that is a download, which I don’t like)

Set Up Companies

Depending on your artifact you may get different companies from the beginning. It seems you always get “My Company”. And then there is a localized CRONUS company (except for the w1 artifact), that can be named “CRONUS USA” or “CRONUS International Inc” or something.

I work for Damage Inc, so that is the only company I want. However, it seems not to be possible to delete the last company. This is what I have automated:

  1. If “Company Zero” does not exist, create it
  2. Delete all companies, except “Company Zero”
  3. Create “Damage Inc”
  4. Delete “Company Zero” (optional – if it distrubs you)

This works the first time (regardless of CRONUS presence), when creating the container. This also works if I run it over and over again (for example when restarting an already created container, or just running some tests on an already started container): I get the same result, just a new “Damage Inc” every time, just as the first time.

Install Rapid Start Package

I install a rapid start package using the automation API. It should be possible to do it from AdditionalSetup.ps1 as well. This takes long time. I see some advantage using the API because I can monitor and control the status/progress in my integration-test-scripts (I could output things from AdditionalSetup.ps1 and monitor that, too).

Rapidstart packages are tricky – by far the most difficult step of all:

  1. Exporting a correct Rapidstart package is not trivial
  2. Importing takes long time
  3. The GUI inside business central (Rapidstart packates are called Configuration Packages) gives more control than the API – and you can see the detailed errors in the GUI (only).
  4. I have found that I get errors when importing using the API, but not in the GUI. In fact, just logging in to the GUI, doing nothing, and logging out again, before using the API makes the API import successful. Perhaps there are triggers being run when the GUI is activated, setting up data?

Run tests!

Finally, you can run your tests and profit!

Use the UI

You can log in to BC with the username and password that you collected above. I am not telling you to do manual testing, but the opportunities are endless.

Stopping

When done, you can stop the container. If run was invoked with “–rm” the container will be automatically removed.

Depending on your architecture and strategy, you may be able to (re)use this container for later use.

Webhooks & Subscriptions

Business Central has a feature called webhooks (in the api it is subscriptions). It is a feature that makes BC call you (your service) when something has updated, so you don’t need to poll regularly.

This is good, but beware, it is a bit tricky.

First M$ has decided BC will only call an HTTPS service. When I run everything on localhost and BC in a container, i am fine with HTTP, actually. Worse, even if I run HTTPS, BC is not accepting my self signed certificate. This sucks! Perhaps there is a way to allow BC to call an HTTP service. I couldn’t find out so now I let my BC Container call a proxy on the internet. That is crap.

Also, note that the webhooks trigger after about 30s. That is probably fine, for production. For automated testing it sucks. Perhaps there is a way to speed this up on a local docker container, please let me know.

Finally, the documentation for deleting a webhook is wrong. In short, what you need to (also) do is:

  1. add ‘ around the id, as in v1.0/subscriptions(‘asdfasdfasdfasdfasf’)
  2. set header if-match to * (or something more sofisticated)

I found it in this article.

Conclusion

This – running NAV/BC inside Docker and automate test cases – is somewhat new technology. There have been recent changes and sources of confusion:

  • NAV rebranding to Business Central
  • Replacing images with Artifacts
  • Multitenant needed from 16.4
  • The onprem vs SAS thing

To do this right requires effort and patience. But to me, not doing this at all (or doing it wrong) is not an option.

Sinkadus om Dungeons & Dragons

Posted by on 2020/09/14 No comments

Uppdatering 2020-09-18: Jag postade en tråd på rollspel.nu och en del intressanta saker framkom.

Vad kunde man läsa i slutet av 80-talet och början av 90-talet i Äventyrsspels hustidning Sinkadus, om det konkurrerande spelet Dungeons & Dragons?

Nedan är vad jag funnit, men notera att:

  • Jag kan ha missat något
  • Jag kan ha utelämnat något obetydligt omnämnande
  • Jag har klippt, och försökt göra det på ett rättvisande sätt
  • Mina egna kommentarer som […kommentar…]

Sinkadus #15: Engelska spel

Advanced Dungeons & Dragons: Detta är den mest populära versionen av det första rollspelet någonsin. Spelet omfattar idag åtta tjocka böcker ofta skrivna på halvdålig engelska. I dessa kan man finna regler för det mesta en rollperson kan råka ut för. Spelets problem är ett rätt trist stridssystem och att man bara konstruerar sin rollperson efter strikta mallar. Det märks att det var det första rollspelet; efterföljarna har lärt sig av AD&Ds misstag. AD&Ds starka sida är att det finns mycket gott om moduler och äventyr till spelet; få har råd att köpa alla.

(Protest! Inte ens med allra godaste vilja kan man kalla AD&D för ett enkelt spel [Sinkadus hade själv kategoriserat det som Enkelt och protesterade mot sig själva]! I sin urpsrungliga version, dvs Players Handbook och Dungeon Master’s Guide, var det på sin höjd medelsvårt, men i och med de otaliga regeltillägg som kommit i Unearthed Arcana, Dragons, två st Survival Guides, Greyhawk Adventures, m.m., har ett redan expansivt spel blivit totalt oöverskådligt! Spelets grundidé, ett snabbt och enkelt stridssystem, har totalt förfelats genom alla de modifikationer som måste göras.)

Sinkadus #15: Insändare med svar

Insändare: Skriv ett kontrakt med Titan Games och ta med Dungeons & Dragons i Sinkadus. I Drakar och Demoner Gigant är det aldrig någon action, inga kraftfulla besvärjelser, inga coola monster. Varför skriver ni alltid så mycket om allt ointressant som politik och religion. Det är väl bra om ni skriver lite om det, men när halva Gigant var en massa utfyllnadsartiklar blir det ju bizzart.

Svar: Titan [utgivaren av D&D i Sverige] har inte rätt att skriva några sådana kontrakt. Och även om det skulle gå, skulle vi inte vara särskillt intresserade. Vad menar ni förresten med “coola monster”? Skicka in ett par förslag så vi vet vad ni menar. Skälet till att vi inte har just de monster ni eftertraktar beror antagligen på att ingen annan vill ha dem. Vi skulle inte ha kunnat sälja så många Gigant till våra kunder om de inte hade uppskattat den typen av artikar som ni uppenbart ogillar. Men – om det bara är monster och häftiga besvärjelser ni är ute efter, så kan ni gott spela Dungeons & Dragons även i fortsättningen.

Sinkadus #16: Insändare med svar

Insändare: Jag vill inte rikta denna kritik mot Äventyrsspel, utan till hela rollspelsgenren och dess utövare. I början var rollspelen mera en sorts regelfixerad grottforskningsövning vilket speglas väl i sådana klassiska mastodontverk som AD&D och dess tidiga biprodukter. I början av 80-talet syntes dock en tydlig förändring till det bättre med klassiker som Runequest och Traveller. Men med rollspelens definitiva intåg i Sverige med den lilla hederliga lådan (första utgåvan av Drakar och Demoner, som jag nog kan hitta i någon dammig byrålåda därhemma) började åtminstone här i Sverige en förändring av hobbyn. […lång fortsättning där insändaren ondgör sig över rollspelare som fastnar i regelboken istället för att rollspela…]

Svar: Tänker ni låta det här stå oemotsagt? Har han rätt, går vi mot en rollspelsvärld där den regelfixerade “vänta-ett-ögonblick-jag-ska-bara-kolla-den-här-modifikationen-i-tabell-si-och-så” helt kommer att dominera?

Sinkadus #17: Insändare

Insändare: Vad menar egentligen valparna med “coola monster”? Jo, att det ska finnas minst 800 mer än sjuka, fullständigt otroliga varelser som endast existerar för att slå ihjäl! Enligt min åsikt ett tämligen meningsklöst livsmål. Pojkar (ev flickor) små, spela ni Dungeons & Dragons och låt Äventyrsspels redaktion slippa bry sina huvuden med vansinniga fantasifoster, och istället få tid att framställa interessanta äventyr som exepelvis Marsklandet, där spelarna kan få välja vad de vill göra och inte bara får hålla på att slå ihjäl “coola monster” och ta deras miljontals “häftiga magiska prylar”.

Sinkadus #17: Insändare och svar

Insändare: Att ha en överblick av engelska rollspel i Sinkadus var bra, men att ha med material till andra än Äventyrsspels egna tror jag inte är så lämpligt. Speciellt inte “slayer-spel” som D&D, hur coola besvärjelser och varelser (Rostmonster! Ha!) det än finns. Det som står i Gigant om politik och religion, som är så “o-coolt” enligt Z är till för att bygga upp en spelvärld, som är mycket “coolare” än ett 30-rums grottkomplex inhysande allehanda “coola” monster. Men om du, Z tycker att rundvandringar i grottor och att döda monster är “coolare”, så kan du, som Red svarade, hålla dig till D&D.

Svar: Om det kan tyckas som att vi hackar mer än en gång på Dungeons & Dragons, så beror det huvudsakligen inte på att det är en konkurrent som har gett ut spelet. Äventyrsspel fick erbjudandet att göra spelet på svenska för ett antal år sedan, men vi tackade nej. Läs vad Åke Eldberg har att säga i sin artikel om moral i rollspel för en farklaring till varför vi inte är så överdrivet förtjusta i D&D.

Sinkadus #18: Etik – Spelsystemets betydelse

Vissa rollspelssystem inbjuder på ett försåtligt sätt till omoral. I det välkända Dungeons & Dragons finns något som kalla “alignment”, vilket ungefär betyder “grundläggande livsinställning”. Det innebär att alla varelser antingen är goda, neutrala eller onda. När man skapar en ny rollperson kan spelaren själv välja hur hans rollperson ska vara. Det är en inbjudan till verkliga dumheter. […] Ett system med förvalda livsinställningar är alltså bedrägligt. Om man är ond eller god kan bara bedömas utifrån vad man göra. Dungeons & Dragons system påminner på ett obehagligt sätt om hur nazisterna definierade judarna som “onda” och behandlade dem därefter.

Sinkadus #18: Insändare och svar

Insändare: Polarstation Z anropar – en tämligen lång och dryg insändare som berättar att D&D är bra, och att D&D Expert har regler för spel utomhus.

Insändare: Jag har med växande irritation sett hur Dungeons & Dragons har blivit något av ett skällsord i Brevspalten. […]

Svar: Jovisst, D&D har det du säger att det har – märkliga monster, skatter, action laddad spelstil, mängder av äventyr – men varför är vi så kritiska mot det. Jo, för att hela spelidén tycks bygga enbart på att man skall slå ihjäl alla de fantastiska monstren och omedlebart stjäla deras sagolika skatter. Skillnaden mot Drakar och Demoner är att du inte har detta inbyggt som en gruläggande förutsättning i spelet. Se även Åke Eldbergs artikel om moral i rollspel i det här numret för en närmare diskussion om det problemet.

Sinkadus #30: Insändare och svar

Insändare: Varför anser så många (särskilt bland äldre rollspelare) att svenska rollspel är “barnsliga” och “töntiga” i jämförelse med engelska och amerikanska? Jag har hört uttryck som Drakaj och Demonej och Drakar och Fiffel. Dessutom ska t ex AD&D vara så JÄTTEAVANCERAT och hela Drakar och Demoner-genren så SIMPEL, regelmässigt sett i alla fall.

Svar: Anledningen till att en massa 17-25-åriga spelare tycker att engelska rollspel är mycket mer avancerade och bättre är för att de är skrivna på engelska. De befinner sig nämligen i en ålder när man skäms över allt som man gjorde när man var yngre, och dessutom inte vill spela samma spel som sina småskyskon – dvs svenska rollspel. Det här är däremot en period som går över – när de blir äldre och mognare skäms de inte längre för att de spelar svenska rollspel. Till den här attityden bidrar många äldre spelare som aldrig har spelat svenska rollspel – de är irriterade på att en massa elvaåringar spelar somma spel som de, och tycker därför att spel som barn kan spela måste vara mycket barnsligare än de amerikanska spel som de själva spelar. Dessutom låter ju allt “häftigare” på engelska. I USA är Advanced Dungeons & Dragons ett utpräglat nybörjarspel för att reglerna är så osofistikerade. Jag har själv spelat AD&D i snart elva år, och jag kan intyga att Expert är ett bättre spel.

Sinkadus #31: Nyheter utifrån

TSR: Det har kommit en ny version av gamla ärevördiga Dungeons & Dragons. Det borde vara en het nyhet för D&D-fansen, men frågan är om inte TSR har tagit i lite för mycket. Spelet är detsamma men förpackningen gör att spelet verkar vara ämnat helt och hållet för rollspelsnybörjarna – med kanske inte helt lyckat resultat. […]

Sinkadus #31: Insändare

Insändare: I och med mitt brev i förra numret hoppas jag verkligen att åtminstone några engelksälskande Drakar och Demoner-hatara fick upp ögonen (om de nu läser Sinkadus) för den enkla sanningen: Drakar och Demoner, inklusive Gigant, är ett av de bästa fantasyrollspelen på marknaden. […] Samtidigt vill jag försvara mig mot dem som kanske fick uppfattningen att jag ogillar AD&D; jag spelade detta spel under en treårsperiod förut, och det gav mig mycket av den kärlek jag har till fantasy. AD&D är annorlunda, är ofta mycket mer sagofullt än Drakar och Demoner, men störs lite av ett ganska tillknäppt regelsystem. Kul var det emertid att möta (små) goblins och hill giants.

Sinkadus #32: Insändare och Svar

Diskussion om vissa rollspel är bättre än andra, och redaktionen medger att det är en personlig uppfattning.

Sinkadus #36: Rollspelens Historia

Detta är en lång (och någorlunda läsvärd) artikel som jag inte alls ska citera i sin helhet.

“D&Ds livssynsystem och uppdelningen av alla tänkade varelser i kaotiska, lagfulla eller neutrala är slutligen hämtat från Michael Moorcooks böcker”

“det kom snart andra rollspel […] som representerade en utveckling bort från D&Ds ganska stereotypa monsterbankande”

“I D&D och de tidiga systemen finns egentligen inte mycket uppmuntran till eller hjälp att skapa en personlighet åt rollpersonen. Han fungerar i stor utsträckning som en sorts stridsmaskin”

“D&Ds system med moralisk livssyn […] hindrar egentligen rollagerandet”

“Titan Games, som översatte Basic Dungeons & Dragons till svenska. Detta var ett försök till en kulturgärning genom att göra rollspelens ursprung tillgängligt för svenska spelare, men eftersom spelsystemet i sig inte hade några nyheter att bjuda på blev det av naturliga skäl aldrig särskilt populärt”

“redan i slutet på 70-talet, faktiskt redan med AD&D som gav spelarna möjlighet att åtminstone då och då få klättra upp ur de trånga fuktiga grottor som hade varit D&Ds hemvist. Redan några av de första rollspelen efter D&D och AD&D frångick dessas groteska erfarenhetssystem, där det enda sättet att bli bättre på var att döda ånga monster och framför allt att hitta stora skatter”

“D&Ds stela stridssystem ersattes också snart av smidigare och mer realistiska regler, med pareringar och färre kroppspoäng [i andra spel då alltså]”

Analys

Vad kan vi göra av detta? Jag tycker nog det är rimligt att påstå att D&D beskrevs på ett övervägande negativt sätt i Sinkadus. En del negativt som skrevs i Sinkadus var insändare. Men det är ändå upp till redaktionen att välja vad som publiceras, och det var unga människor som skrev. Jag tycker nog att de fulare sakerna till stor del inte var insändare.

Min bakgrund

Jag spelade både Drakar och Demoner (mest Expert och DoD-91) och Dungeons & Dragons (Titan Games och senare AD&D 2e). Jag spelade också många andra rollspel, mest då: Western, Khelataar, Skuggornas mästare och Rolemaster. Där jag växte upp var allt lika töntigt. Ingen brydde sig om jag spelade DoD eller D&D. Det var inte heller så att våra äventyr, berättelser, karaktärer var speciellt annorlunda i DoD eller D&D. Det var reglerna som skiljde: inte huruvida det var grottor, monsterdödande, karaktärsutveckling, rollspel, problemlösning eller politik. Men de kampanjerna som överlevde längst, som gjorde störst intryck, vars karaktärer vi minns idag, och som vi fortfarande kan prata om, de spelade vi i D&D och AD&D.

Nybörjare

När man läser Sinkadus får man en blandad bild av synen på nybörjare. Nybörjare används som ett skällsord om målgruppen för en ny D&D-box (S#31). De som inte gillar Gigant (politik och religion tydligen, själv minns jag Gigant lite som en besvikelse) kan lika gärna spela D&D (S#15). Och samtidigt förstår Äventyrsspel att nybörjare – unga svenska spelare som har svårt för engelska – är deras kärngrupp (S#30).

Moral

Att i en artikel om Etik jämföra alignment-systemet i D&D med nazisterna (S#18) verkar lite godhetssignallerande (redan i början av 90-talet), och när det kommer från bolaget som sedan släppte Kult (appropå Etik) förefaller det mest vara hyckleri. För egen del har systemet med livsåskådningar i D&D aldrig varit viktigt, men spelets rötter i Moorcook är sådana (S#36).

Så här 2020 kan man ju dessutom reflektera över begreppet “svartfolk”.

Karaktärsutveckling

Jag har faktiskt svårt att förstå vad som menas med “I D&D och de tidiga systemen finns egentligen inte mycket uppmuntran till eller hjälp att skapa en personlighet åt rollpersonen”, eller rättare sagt: vad finns i DoD Expert?

I DoD Expert, om man börjar som Jägare eller Stråtrövare, och som sin del av karaktärsutvecklingen vill bli dubbad till Riddare, så är det tämligen oöverstigligt att lära sig Rida, Lans och Slagsvärd. Även en Krigare som råkade börja spelet med en stridsyxa eller morgonstjärna har svårt att någonsin blir Riddare med svärd.

I DoD Expert, om man skulle vilja spela en karaktär som Elric på omslagen till DoD, så överlever man inte länge. Utan rustning på armar och ben är man ett lätt offer för vilket gäng svartnissar som helst. Och den som vill spela en barbar utan rustning har ännu mindre chans.

I DoD Expert är det svårt att föreställa sig den magiker som byter magiskola efter några år.

Därmed inte sagt att D&D eller AD&D är perfekt. Det tycker jag inte.

Grottor

Temat om grottäventyr återkommer. Men som S#36 själv konstaterar, redan i slutet av 70-talet fanns regler för spel ovan jord – dvs innan DoD alls var släppt. Wilderness Survival Guide släpptes 1986 (alltså samtidigt som S#5, och före allt ovanstående) och är en av de böcker som S#15 ondgör sig över.

Sinkadus är för övrigt inte direkt befriat från äventyr i grottor.

Monster

Jag ägde D&D Basic + Expert och DoD Monsterboken I & II. Jag tycker det är hårklyverier att säga att fokus på monster är så olika. Äventyrsspels Monsterböcker är för övrigt bland det finaste de producerat.

Erfarenhetspoäng (och OSR)

Enligt S#36 är erfarenhetssystemet i D&D groteskt. Spelets ursprung handlar om utforskning, överlevnad, begränsade resurser och att finna förlorade rikedomar och ta hem dem till civilisationen. Att det sedan fanns lika många spelstilar som spelgrupper är en annan sak.

Det är betydligt enklare att belöna sina spelare med EP för något annat i D&D, än det är att ändra stridsreglerna i DoD Expert så att det går att spela Conan eller Elric.

Fokus på strid

D&D beskrivs nedlåtande som actionladdat (S#18) och monsterbankande (S#36). DoD Expert har följande mekanismer för strid som saknas i D&D:

  • Parader
  • Brytvärde
  • Kroppdelar (med kroppspoäng, att träffa, och att ha rustning på)
  • Olika storlekar på sköldar
  • Olika färdigheter för olika vapen
  • STY-grupp (heter det så?) på vapen

Varför skulle D&D vara mer actionladdat, när DoD Expert lägger så mycker mer fokus på detaljer i strid?

Nedlåtande Arrogans

Det här med dålig engelska (S#15), märkliga monster (S#18), osofistikerat utpräglat nybörjarspel (S#30), kulturgärning (S36) förefaller bara vara arrogant högmod – som går före fall. Och då har jag bara citerat redaktionen, inte vad som passerade på insändarsidorna.

Slutsats

Som jag skrev så växte jag upp i en miljö där ingen brydde sig om det var Drakar och Demoner eller Dungeons & Dragons.

Men jag har senare, lång senare, upplevt att många spelare har en ganska negativ syn på Dungeons & Dragons utan att de för den skulle har spelat det särskillt mycket, eller alls.

Jag förvånas över rollspelare som lyfter fram vikten av “seriöst rollspel och karaktärsutveckling” inte ser att att enkelt spelsystem som inte är i vägen är precis vad som möjliggör detta.

Jag kan förstå, och se, Äventyrsspels utmaningar under 90-talet att både behålla gamla spelare, rekrytera nya spelare, och konkurrera med både svenska och utländska alternativ. Nu vet vi hur det gick. Äventyrsspel gick under, och för Drakar och Demoner kom en lång sorglig ökenvandring, som varumärket aldrig verkar hämta sig från. Vad som hade behövts, istället för att snacka skit om Dungeons & Dragons, hade varit ett tydligt fokus på att göra Drakar och Demoner till en bättre produkt och en tydlig uppfattning om sin målgrupp. Chronopia och Kult visar att man missade båda målen.

Kan det vara så att dessa texter i Sinkadus har haft en stor påverkan på svenska rollspelares uppfattning om Dungeons & Dragons, och att de till stor del är missledda? Nu är detta 30 år gamla texter, men fördomarna, missuppfattningarna och intolerans snarare än nyfikenhet mot de som gillar andra spel lever kvar. Tycker jag.

Vi rollspelare i detta lilla land borde kunna lite bättre. Eller?

OSR Melee Weapon Damage House Rules

Posted by on 2020/09/12 1 comment

If you look at old versions of D&D (OD&D, B/X, BECMI/Rules Cyclopedia, 1e, 2e) they have lists of weapons with varying damage (or not – varying damage is optional rule in some versions). Newly written OSR games as well as newer versions of D&D have their own tables – and bring some innovation as well.

If you start looking into these tables there are inconsistensies, making some choices objectively inferior to others (examples below). I don’t like that. I prefer when there are both pros and cons – risk and reward – in a choice.

Proposed House Rules

Different classes have different capability to do damage, but to reach that potential a proper weapon is required.

1d41H Basic weaponsAllDagger
1d62H Basic weaponsAll but RogueStaff
1d61H Standard weaponsAll but WizardWarhammer
1d82H Standard weaponsAll but Wizard, RogueSpear
1d81H Martial WeaponFighter, Elf, Dwarf, RogueLong Sword
1d102H Martial WeaponFighter, Elf, Dwarf2H Sword

Basic weapons are mundane everyday items (dagger, staff, club).
Martial weapons are rather large AND use blades for extra effect.

Compared to BECMI/Rules Cyclopedia it really changes very little in most cases.

Extra Features

To make weapons different each weapon can have an extra features reflecting its nature. Typically:

  1. Weapons can be versatile (used in 1 or 2 hands) OR they have an extra feature
  2. Non-martial weapons can have extra features (to compensate for lower damge)
Bastard SwordM12
Battle AxeM12
ClubB12
DaggerB11d10 damage when used for sneak attack or at DMs descretion at intimate range
FalchionM11d6+1 damage (slightly better than 1d8)
Great AxeM21d12 damage, disadvantage to initiative
Hand AxeS1Can be thrown
Long SwordM1+1 damage when rolling 1.
Pole WeaponM2Can be used from behind ally.
Advantage to initiative 1st round.
RapierM1May choose to roll initiative with advantage, for 1d6 damage
Short SwordS11d8 damage when used for sneak attack or at DMS descretion at intimate range
SpearS12Advantage to initiative.
Can be thrown.
StaffB12Advantage to initiative when 2H
Two Handed SwordM2+1 damage when rolling 1
WarhammerS12

Above list is just an example. Any weapon IS a basic, standard or martial weapon.

Practical Use

These rules set the wielder in focus, not the weapon:

  1. Can the character use the weapon at all
  2. Does it match the characters potential (a martial weapon for a fighter)
  3. Is it 1H, 2H or choice
  4. Are there any extre features

The features could depend on the wielder. A dwarf may use it as a throwable 1H hand axe, but a halfling may use it as a 1-2H axe.

Weapon Quality and Damage

Some weapons are inferior or superior.

  • Inferior: -1 damage on best two rolls (5,6 on 1d6)
  • Low quality: -1 damage on best roll (8 on 1d8)
  • Standard quality
  • High quality: +1 damage on natural 1. (10x price)
  • Superior quality: +1 damage on natural 1,2. (100x price)

This should be compared to magic weapons doing +1 always. Bad quality could reflect bad maintenance and condition.

Shields

Shields give +1 Armor Class (some new OSR games may be different). I employ the house rule that a shield can be sacrificed to save the wielder from a blow that would otherwise be deadly.

Two Weapons

I employ the house rule that anyone fighting with two weapons gets advantage on initiative. If the two dice show same result, the attack is made with the 2nd weapon (only). If the second weapon is not smaller than the first, that attack is made with -2.

After playing a bit of 5th edition, I think multiple attacks is an annoyance and a mistake. Then there is always the question whether you can actually effectively use two weapons at the same time, and whether you can use your off-hand effectively. My proposed rule can be thought of as “it does not hurt to hold a weapon in your off-hand, occationally it can be your best option”.

Initiative

The standard BECMI / Rules Cyclopedia rules state that the wielder of a 2H weapon loses initiative. I see two problems with that:

  1. There is already a penalty of using a 2H weapon: no shield
  2. I think of 2H weapons as fast (staff, spear) or long reach (pole arms), which should be rather good to initiative

I am experimenting with using advantage/disadvantage (roll 2 dice, use the best/worst) for initiative. If you find that offensively 5th edition, you can of course use +2/-2 instead.

Background

The following paragraphs are just background and reasoning behind the house rules proposed above.

Expected Damage

If you do 1d8 damage and want to do better, what do you prefer? Table shows hits required to kill an enemy with 4, 10 or 11 HP.

4HP10HP11HP
1d81.422.762.97
1d4+31.002.192.38
1d6+21.172.282.44
1d8+11.252.372.51
1d101.332.362.59

Why is 1d4+3 better than 1d10? Why is 1d6+2 better than 1d8+1? This is a bit counter intuitive. They all do 5.5 HP damage, on average (except 1d8 doing 4.5). The reason is that an opponent with 4HP is always killed in one hit with 1d4+3, but survives 3/10 with 1d10. And after a few blows, any enemy may end up with 4 remaining HP.

An abstract system

It is important to recognise that D&D in general, OSR in particular, has an abstract and simplified set of rules. Not everything is in the system. A perfect simulator would need to consider:

  • Manufacturing cost
  • Durability
  • Need for maintenance
  • Cost of maintenance
  • Efficiency against different armors, and perhaps enemies
  • Speed and range
  • Training required
  • Strength required

At some point you get to the point where you reason that a more experienced fighter will beat anyone even if he is armed with only a broken chair and a handful of sand. Perhaps 1d6 for all weapons is not so bad after all?

Inconsistencies

Here are a few strange things

  • A Bastard sword can be used in one or two hands, for 1d6+1 or 1d8+1 damage respectively. That means it is a better 1H weapon than the 1d8 longsword AND a better 2H weapon than the 1d10 2H-sword. It is just better.
  • In BECMI a dwarf can use a 2H Broad Axe for 1d8 damage. Or a 1H Long Sword for 1d8 damage. There is no advantage using an axe (initiative is lost and a shield can not be used).
  • A hand axe is cheaper than a short sword, but can also be used as a thrown weapon.
  • Spears are generally underestimated

Infinity Whisky Bottle – Good Idea?

Posted by on 2020/09/02 No comments

I am not going to explain what an Infinit Bottle is. Use Google.

And I am not ready to just planlessly add my whisky remains to a bottle either. But I will experiment a little.

Motörhead + Storm

Motörhead and Storm are two whiskies in the lower part of my head-to-head ranking. I am almost out of of both of them and I am bored of them, so they could be good candidates for going into an Infinity bottle. Storm is too dry and bodyless. Motörhead is too sweet. Do they taste better if mixed?

Color: quite obviously the mix falls between the pale Storm and the dark Motörhead. I am not goint to argue that the color of the mix is more appealing.

Aroma: Motörhead has a very soft bourbon aroma. Storm has little aroma, a bit chemical. The mix is, quite obviously more balanced. It is much more promising than Storm, and not as in-your-face as Motörhead.

Taste: Being kind, Storm does not taste so bad. For being a blend it has some quality. Over to Motörhead, there are some very sweet dominant flavours that are not entirely nice. How about the 50/50-mix? Well, not too surprisingly it has less of the bad stuff than both the original whiskies. But it also has less of their characters – for good and for bad. In this particular case, my opinion is that it was more good than bad.

Conclusion: MotörStorm is better than both Motörhead and Storm

PHP validation of UTF-8 input

Posted by on 2020/08/18 No comments

Last weeks I have done some PHP programming (my web hotel where I run wordpress supports PHP, and it is trickier to run Node.js on a simple web hotel). I like to do input validation:

function err($status,$msg) {
  http_response_code($status);
  echo $msg;
}

if ( 1 !== preg_match('/^[a-z_]+$/',$_REQUEST['configval']) ) {
  return err(400,'invalid param value: configval=' . $_REQUEST['configval']);
}

Well, that is good until I wanted a name of something (like Düsseldorf, that becomes D%C3%BCsseldorf when sent from the browser to PHP). It turned out such international characters encoded as Unicode/UTF-8 can not be matched/tested in a nice way with PHP regular expressions.

PHP does not support UTF-8. So ü in this case becomes two characters, neither of them matches [A-Za-z] or [[:alpha:]]. However, PHP can process it as text, use it in array keys, and output valid JSON without corrupting it, so not all is lost. Just validation is hard.

I needed to come up with something good enough for my purposes.

  • I can consider ALL such unicode characters (first byte 128+) valid (even though there may be strange characters, like extra long spaces and stuff, I don’t expect them to cause me problems if anyone bothers to enter them)
  • I don’t need to consider case of Ü/ü and Å/å
  • I don’t need full regexp support
  • It is nice to be able to check length correctly, and international characters like ü and å counts as two bytes in PHP.
  • I don’t need to match specific characters in the ranges A-Z, a-z or 0-9, but when it comes to special characters: .,:,#”!@$, I want to be able to include them explictly

So I wrote a simple (well) validation function in PHP that accepts arguments for

  • minimum length
  • maximum length
  • valid characters for first position (optional)
  • valid characters
  • valid characters for last position (optional)

When it comes to valid characters it is simply a string where characters mean:

  • u: any unicode character
  • 0: any digit 0-9
  • A: any capital A-Z
  • a: any a-z
  • anything else matches only itself

So to match all letters, & and space: “Aau &”.

Some full examples:

utf8validate(2,10,’Aau’,’Aau 0′,”,$str)

This would match $str starting with any letter, containing letters, spaces and digits, and with a length of 2-10. It allows $str to end with space. If you dont like that, you can do.

utf8validate(2,10,’Aau’,’Aau -&0′,’Aau0′,$str)

Now the last character can not be a space anymore, but we have also allowed – and & inside $str.

utf8validate_error

The utf8validate function returns true on success and false on failure. Sometimes you want to know why it failed to match. That is when utf8validate_error can be used instead, returning a string on error, and false on success.

Code

I am not an experienced PHP programmer, but here we go.

function utf8validate($minlen, $maxlen, $first, $middle, $last, $lbl) {
  return false === utf8validate_error($minlen, $maxlen,   
                                      $first, $middle, $last, $lbl);
}

function utf8validate_error($minlen, $maxlen, $first, $middle, $last, $lbl) {
  $lbl_array = unpack('C*', $lbl);
  return utf8validate_a(1, 0, $minlen, $maxlen,
                        $first, $middle, $last, $lbl_array);
}

function utf8validate_utfwidth($pos,$lbl) {
  $w = 0;
  $c = $lbl[$pos];
  if ( 240 <= $c ) $w++;
  if ( 224 <= $c ) $w++;
  if ( 192 <= $c ) $w++;
  if ( count($lbl) < $pos + $w ) return -1;
  for ( $i=1 ;$i<=$w ; $i++ ) {
    $c = $lbl[$pos+$i];
    if ( $c < 128 || 191 < $c ) return -2;
  }
  return $w;
}

function utf8validate_a($pos,$len,$minlen,$maxlen,$first,$middle,$last,$lbl) {
  $rem = 1 + count($lbl) - $pos;
  if ( $rem + $len < $minlen )
    return 'Too short';
  if ( $rem < 0 )
    return 'Rem negative - internal error';
  if ( $rem === 0 )
    return false;
  if ( $maxlen <= $len )
    return 'Too long';

  $type = NULL;
  $utfwidth = utf8validate_utfwidth($pos,$lbl);
  if ( $utfwidth < 0 ) {
    return 'UTF-8 error: ' . $utfwidth;
  } else if ( 0 < $utfwidth ) {
    $type = 'u';
  } else {
    $cv = $lbl[$pos];
    if ( 48 <= $cv && $cv <= 57 ) $type = '0';
    else if ( 65 <= $cv && $cv <= 90 ) $type = 'A';
    else if ( 97 <= $cv && $cv <= 122 ) $type = 'a';
    else $type = pack('C',$cv);
  }

// type is u=unicode, 0=number, a=small, A=capital, or another character

  $validstr = NULL;
  if ( 1 === $pos && '' !== $first ) {
    $validstr = $first;
  } else if ( '' === $last || $pos+$utfwidth < count($lbl) ) {
    $validstr = $middle;
  } else {
    $validstr = $last;
  }

  if ( false === strpos($validstr,$type) ) {
    return 'Pos ' . $pos . ' ('
         . ( 'u'===$type ? 'utf8-char' : pack('C',$lbl[$pos]) )
         . ') not found in [' . $validstr . ']';
  }
  return utf8validate_a(1+$pos+$utfwidth,1+$len,$minlen,$maxlen,
                        $first,$middle,$last,$lbl);
}

That is all.

Tests

I wrote some tests as well.

$err = false;
if (false!==($err=utf8validate_error(1,1,'','a','','g')))
  throw new Exception('g failed: ' . $err);
if (false===($err=utf8validate_error(1,1,'','a','','H'))) 
  throw new Exception('H should have failed');
if (false!==($err=utf8validate_error(3,20,'Aau','Aau -','Aau','Edmund')))
  throw new Exception('Edmund failed: ' . $err);
if (false!==($err=utf8validate_error(3,20,'Aau','Aau -','Aau','Kött')))
  throw new Exception('Kött failed: ' . $err);
if (false!==($err=utf8validate_error(3,20,'Aau','Aau -','Aau','Kött-Jan')))
  throw new Exception('Kött-Jan failed: ' . $err);
if (false!==($err=utf8validate_error(3,3,'A','a0','0','X10')))
  throw new Exception('X10 failed: ' . $err);
if (false!==($err=utf8validate_error(3,3,'A','a0','0','Yx1')))
  throw new Exception('Yx1 failed: ' . $err);
if (false===($err=utf8validate_error(3,3,'A','a0','0','a10')))
  throw new Exception('a10 should have failed');
if (false===($err=utf8validate_error(3,3,'A','a0','0','Aaa')))
  throw new Exception('Aaa should have failed');
if (false===($err=utf8validate_error(3,3,'A','a0','0','Ax10')))
  throw new Exception('Ax10 should have failed');
if (false===($err=utf8validate_error(3,3,'A','a0','0','B0')))
  throw new Exception('B0 should have failed');
if (false!==($err=utf8validate_error(3,3,'u','u','u','äää')))
  throw new Exception('äää failed: ' . $err);
if (false===($err=utf8validate_error(3,3,'','u','','abc'))) 
  throw new Exception('abc should have failed');
if (false!==($err=utf8validate_error(2,5,'Aau','u','Aau','XY')))
  throw new Exception('XY failed: ' . $err);
if (false===($err=utf8validate_error(2,5,'Aau','u','Aau','XxY')))
  throw new Exception('XxY should have failed');
if (false!==($err=utf8validate_error(0,5,'','0','',''))) 
  throw new Exception('"" failed: ' . $err);
if (false!==($err=utf8validate_error(0,5,'','0','','123'))) 
  throw new Exception('123 failed: ' . $err);
if (false===($err=utf8validate_error(0,5,'','0','','123456')))
  throw new Exception('123456 should have failed');
if (false===($err=utf8validate_error(2,3,'','0','','1'))) 
  throw new Exception('1 should have failed');
if (false===($err=utf8validate_error(2,3,'','0','','1234'))) 
  throw new Exception('1234 should have failed');

Conclusions

I think input validation should be taken seriously, also in PHP. And I think limiting input to ASCII is not quite enough 2020.

There are obviously ways to work with regular expressions and UTF8 too, but I do not find it pretty.

My code/strategy above should obviously only be used for labels and names where international characters make sense and where the form of the input is relatively free. For other parameters, use a more accurate validation method.