MiniScript: lightweight scripting language for your game

Yes. I agree with you 100% on this point. I was talking from my point of view and forgot who the end user is... I think that having fewer options is the best option in this case.

 

I have found a bug that can crash the game. I have been able to reproduce it in one of the example scenes.
To reproduce the error you have to create a script with a very large loop:

for i in range(0, 100000000000000000)

   test = "a"

end for

print("end") 

it seems that somehow even if it exceeds the maximum time (0.01) it is still processed.
It happens with a single call to RunUntilDone(). That is, without being in a loop or in the update function.

Is there any way to prevent this?

 

I'll look into this today and get back to you!

 

OK. The problem has nothing to do with the loop; it's the (attempted) creation of a list with
100 quadrillion elements in it (which is what the range() call does here). It fills up memory and goes boom. And unfortunately, takes the whole Mono environment — including the IDE — down with it. You might expect Mono to be a little smarter about memory management than that, but no.

So I guess MiniScript needs to be smarter about it. Because ideally there shouldn't be any code you can type into a MiniScript context that brings down the host app. I will look for a solution.

EDIT: OK, here's at least a first step. Open up MiniscriptInstrinsics.cpp, and find the code for the range() function (just search for

 Intrinsic.Create("range")

). You will find the for-loop that builds the requested list:

Code (CSharp):

1.         for (double v = fromVal; step > 0 ? (v <= toVal) : (v >= toVal); v += step) {
2.             values.Add(TAC.Num(v));
3.         }

Replace that code with this:

Code (CSharp):

1.     try {
2.         for (double v = fromVal; step > 0 ? (v <= toVal) : (v >= toVal); v += step) {
3.             values.Add(TAC.Num(v));
4.         }
5.     } catch (SystemException e) {
6.         // uh-oh... probably out-of-memory exception; clean up and bail out
7.         values = null;
8.         throw(new RuntimeException("range() error", e));
9.     }

Now when you try some shenanigans of this sort, you will get a range error (MiniScript exception) rather than crashing the app.

Note that there will still be a substantial delay while MiniScript tries to create a list with 100 quadrillion elements. That's not ideal, but it's a pathological case, and certainly more acceptable than crashing.

This change (or something similar) will be in the next MiniScript update. I'll also try to look for and button up any other ways a malicious user could cause such a problem. Thank you for bringing it to my attention.

 

thanks for your quick response!

It may not be ideal, but perhaps you could limit the maximum number of elements that can be on a loop? since players do not need to create such large loops.

In my case, this issue it is a bit more problematic since I am using it in a multiplayer environment where the scripts are shared.

In any case I have to say that I am very satisfied with miniscript and how extensible it is. You have done a great job!

EDIT: wow! this has been fast! thank you.

 

humm, I'm sorry, but it seems I had not tested it correctly before. I've tried it again in the example scene with the change you've made and it's still crashing me.
I'm developing on a mac.

 

Ugh, no, I'm the one who didn't test it correctly. I have an isolated, pure C# test environment where I do most of my low-level MiniScript work. That showed the problem originally but handled it gracefully after the change. But the Unity environment is different — an older version of Mono, for one thing — and you're right, it still crashes there.

All right, if you're willing to put a hard limit on what range() will do, then insert this code right before the new try block:

Code (CSharp):

1.         double count = (toVal - fromVal) / step;
2.         if (count > 1E6) {    // bail out if requested list is too big
3.             return new Intrinsic.Result(new ValList());
4.         }

This will make range() return an empty list for any set of parameters that would produce more than a million elements. In my testing (also on a Mac), this solves the problem. I don't love it as a long-term solution, but it should at least get you going for now.

 

thanks! that worked. Since the limit is really high, it will not interfere with the players' scripts.

 

I'm not sure but I think I've found another problem. It is not a serious error, but it causes a crash in miniscript.

Here is the example code that I used to reproduce the error:

val1 = null

val2 = 0

print("hello1")

if not val1 or val2 then

    print("hello2")

else

    print("hello3")

end if

print("END")

This causes this crash.
https://imgur.com/BBjw3a4

It seems that there is a problem when checking if one of the values is null, and it does not enter the catch section in the interpreter.RunUntilDone() part.

Is this behavior intentional? I have tested this code in the example scene.

 

Yikes! Definitely not intentional. Great catch!

I have a list of (mostly small) improvements to MiniScript I've been sitting on. This is the first serious bug (other than the over-trusting range() function a few posts up) that anyone has found in quite a while. So it's enough to spur me to wrap up my other changes, and post a MiniScript update. I'll try to get that out in the next week or two.

 

great to hear that! I wait for the update. Thanks for the support.

 

@JoeStrout I was running into a situation where when I call

Code (csharp):

1.  
2.    try
3.    {
4.        interpreter.Compile();
5.    }
6.    catch (MiniscriptException e)
7.    {
8.  
9.    }
10.  

I can't seem to get an exception to throw on Compile().

An example miniscript file that I was using, with purposefully blatant errors, is:

Code (csharp):

1.  
2. gTitle="Hello world"
3. akj)
4. {
5. OnGameBegin = function()
6.  
7. //end function
8.  

But for some reason I wasn't getting any exceptions thrown. Is there something I'm doing wrong?

 

Maybe? They're certainly thrown for me, because that's how the demos display compile errors. So, somehow your project is different. Not sure exactly what that might be, though.

If you can put together a small project demonstrating the problem, PM it to me and I'll have a look.

 

So, to my understanding, MiniscriptInterpreter.cs has the following code block:

Code (csharp):

1.  
2. public void Compile()
3. {
4.    ...
5.    parser = new Parser();
6.    try
7.    {
8.        parser.Parse(source);
9.        ....
10.    }
11.    catch (MiniscriptException mse)
12.    {
13.       ReportError(mse);
14.    }
15. }
16.  

This tells me that if the compiler Parser throws an error, the MiniscriptException is caught, but it doesn't get rethrown, so my try/catch for my intepreter.Compile() will never get handled.

This also tells me that Miniscript is intended to handle MiniscriptExceptions internally, and the way errors are reported are via the errorOutput TextOutputMethod. This is great for abstraction and all, but what if I want to know if it's specifically a Compiler error? Is it possible to have an ExceptionOutputMethod so that client code can dig into the exception? Or perhaps this is against your design philosophy? Thoughts?

 

Ah yes, right you are. But the exception is passed to ReportError (which, as I recall, you're intended to override). So you can examine the error there and see what sort of exception it is.

 

I think I've found another bug, tested in the example scene. It could also be called inconsistency.

According to the documentation, the methods that do not accept parameters do not need to invoke them with ().
But for example this does not work, it throws an error of "too many arguments":

Code (CSharp):

1. print("Spam".upper.indexOf("A"))

But this works:

Code (CSharp):

1. print("Spam".upper().indexOf("A"))

It seems that the () are not required if the child function has not parameters passed to it, but fails if the child function has parameters as in this case.

Is there any solution to this or should I call the methods always with ()?

 

I am sorry if this was answered already, but I didn't have time to read through the whole thread.
Anyway, is it possible to save the state of a MiniScript interpreter instance, and load that state in a later session - or is this actually not a feasible?

 

Hmm, you're right. That is not intended. I've added it to my test suite and will get a solution ASAP (I have a few other things in the hopper to be released at the same time).

Until then, yes, you'll have to use the empty parentheses.

 

It's feasible, but you're the first to ever want it as far as I know. You would just need to walk the state of the interpreter object (in particular, its virtual machine) and save it out. Most of that data is in the form of MiniScript values, which are wrappers for basic types (string, double), or lists or dictionaries of same.

That serializing/deserializing isn't something that MiniScript provides out of the block, but you could certainly add it with a bit of effort.

 

Hi, how much is your asset integrated into unity? Could i possibly use it standalone for some server side scripts?

 

Thanks for the answer. My question is aiming towards saving an NPC's state in some kind of RPG or adventure, where the NPCs can have more complicated scripts, and where it's possible to save anytime during gameplay. Would there be a better way to do this?

 

It is neatly encapsulated and works fine outside of Unity. I myself have a command-line Mono project I use to do most of the MiniScript development. So it will work fine server side.

 

I would recommend making the NPCs event-driven. So rather than have scripts designed to be running all the time, they should be set up to run small bits of code in response to something happening (possibly including a periodic tick of the clock), and return very quickly.

These scripts could still have persistent state, which would appear to your Unity code as a ValMap, but you'd only have to save and restore that map — you wouldn't have to worry about trying to save the entire state of the virtual machine (including the call stack, program counter, etc.).

 

Yeah, sounds sensible. My thinking was the modding-ability makes it hard to predict what the script will look like, so "anything is possible in a state".

 

For our game, we are looking to have an in-game code editor so people can write their own AI, very similar to Robo Reindeer Rumble. We haven't pulled the trigger on miniscript yet, but it looks to be fairly ideal as a scripting language for players.

Has anyone made an attempt at creating an in-game code editor for miniscript?

 

I spend a couple of weeks on writing one for my game. It's no easy task but I am getting there. What I have so far is a code editor with the following functionality (all WIP):

-Syntax Highlighting
-Auto Indent and brackets
-Block folding
-undo, redo, copy, paste, etc.
-crude auto-complete

I was planning to post the package here once finished, but as I said it's not easy and it's on hold right now due to other obligations so might take a month or two before it's complete.

The base editor is based on text mesh pro text input and the work of the highlighter is done off the main thread to avoid CPU spikes.

here's what it looks like

 

Hi Joe,

I found a possible bug in Miniscript and found the fix for it locally so I thought I would ping you.

Code like this

dot.sequence.call(param)

was not working correctly for me.

The code I was getting back

Code (CSharp):

1.    
2. [0]: "_0 := call param with 0 args"
3. [1]: "push param _0"
4. [2]: "_1 := call dot[sequence] with 1 args"
5. [3]: "_2 := call _1[call] with 0 args"
6. [4]: "_ := _2"
7.  

It was trying to call dot[sequence] with the param, and not call with the param.

I tracked the issue down to how ParseDotExpr works, and that the call to evaluate chained sequences should be a FullyEvaluate, not ParseCallArgs like it is right now. The ParseCallArgs sees the LParam since the identifier in between the last value and the parameter list is already consumed by RequireToken, so it acts like the call is on the previous ValSeqElem.

 

Would you be willing to share what you have so far? Maybe the community can help finish it?

 

You sure did! This is in fact the same bug that @LoadingHome found last week. I have a fix (pretty much the same one you found) working and tested here. I'm planning to wrap this up, including a handful of other minor improvements, and push it out today.

Thank you!
- Joe

EDIT: MiniScript Version 1.3 has been submitted to the Asset Store. I'll announce here as soon as it is approved.

 

@JoeStrout I should firstly say I've found MiniScript to be very impressive. I've been working with MiniScript as the end user of a game that incorporates it.

I have, however, encountered a few issues (including the above bug on the subject of empty parentheses).

Could you perhaps explain what appears to be happening with the following:

Code (JavaScript):

1. mylist = []
2. if mylist then
3.   print("It's true") // doesn't print; because mylist is empty
4. end if
5.  
6. if not mylist then
7.   print("It's false") // also doesn't print, yet mylist is empty!
8. end if

My understanding from the MiniScript manual is that empty strings/maps/lists are supposed to evaluate as false, and yet they do not do so when using the not keyword. However, they do so when evaluated via else against the truth of the value:

Code (JavaScript):

1. mylist = []
2. if mylist then
3.   print("It's true") // doesn't print; because mylist is empty
4. else
5.   print("It's false") // prints, because mylist is empty
6. end if

This seems like a strange discrepancy.

Another issue I've noticed is that str/map/list.values seems to throw a 'not found in map'.

Finally (apologies for all of this coming out in one go!), the MiniScript 1.2 manual on page 23 states list.remove takes a value (x), when in fact it appears to take an int index

I suspect these issues may be resolved already with 1.3, but I thought it important to flag them nonetheless.

 

Thanks for this feedback! I wish you had posted it yesterday, but I'm glad to get it nonetheless.

Dang. You're right! This appears to be another not-bug that was not exposed by my test suite. It is now, and I'll get it fixed for the next update.

Hmm, this one seems to be working OK for me. Possibly it's fixed in MiniScript 1.3. Can you give me an example?

Ah, right you are again. This was correct in the Quick Reference, but wrong in the manual. It is indeed an index.

Thank you for reporting this. You've just made MiniScript better! Unfortunately I can't resubmit version 1.3, but I have a handful of other improvements in mind, so I may hang on to these for a few more weeks and roll them out with 1.4.

What's the game you're playing that uses MiniScript?

 

An awesome game by LoadingHome

Code (JavaScript):

1. list = range(0,5)
2. print(list.values)

On your MiniScript Playground and Demo pages, this produces key not found: 'values' not found in map. I previously figured this was because those two pages are perhaps using MS <1.2, but the same error is returned when trying to use .values in game (which uses 1.2).

 

Hi; i'm also playing around with miniscript. I need to have methods (functions) which only work in a certain vm. I could use hostData and reject the call if it is not matching, but i would prefer to have non static intrinsic functions. Would it be possible for you to add this feature?

My preferred solution would be to instantiate an interpreter and maybe load default intrinsic functions into it - of course it would cost some memory.

kind regards
michael

 

OK, it works in the current (1.3) version of MiniScript, but it's true that I haven't updated the web builds in a while. I'll try to put that on my to-do list. In the mean time, be sure to point out to @LoadingHome that there's a new version of MiniScript he might want to update in his game.

 

Hmm. The way intrinsics work (each one referred to internally by a unique numeric ID), this is a bit tricky.

But you don't have to use intrinsics; you could instead use ordinary global functions, which you poke into the user's context via interpreter.SetGlobalValue. (Get these, for example, by first compiling a script that defines all the functions, and then pulling them out with GetGlobalValue.) To the user, this is the same effect, but it would allow you to supply different global functions in different contexts.

Of course if your intrinsics need to do things that can't be done in ordinary MiniScript code, that may not be sufficient.

OK then, how about this: define all your intrinsics, but override them as needed by calling SetGlobalValue. You could just set them to null, or set them to some function that prints "Invalid context for this intrinsic" or whatever. This would completely prevent the scripts from accessing them, as there would be no reference bound to them.

 

MiniScript version 1.3 is now available in the Asset Store! Changes in this release:

• Statements beginning with not (which only make sense in a REPL context) now evaluate correctly.
• Null values now properly work in boolean expressions involving and, or, and not.
• Chained calls are now properly invoked even when some of those calls do not have parentheses; fixes a failure to parse, for example, s.upper.indexOf("R") where s is some string.
• Empty strings and null strings are now correctly compared; fixes a bug that, among other things, broke the "FizzBuzz" example in the manual.
• Added intrinsic function ceil(x): returns next whole number equal to or less than x.
• Added intrinsic function floor(x): returns next whole number equal to or greater than x.
• Added intrinsic function list.sort: sorts the given list in place. Optionally, specify the name of a key to sort by — e.g. list.sort("name") — and it will sort all elements by that key (useful when you have a list of objects, for example).
• Added intrinsic function yield: makes the main loop in the interpreter (MiniscriptInterpreter.RunUntilDone) return immediately. In a game context, for example, this would typically pause the script until the next frame.
• The indexOf intrinsic function now takes an optional 'after' parameter. If given, the search begins after the indicated key. Useful for looping over all matches (by passing in the previous value returned). Works for strings, lists, and maps.

This is a recommended update for all users.

I'm already working on the next release, which will fix use of the not operator with lists and maps, and also add a brand new syntax-coloring code editor you can use in your games! But I still recommend you upgrade to 1.3 now.

And thanks to all you who are using MiniScript in your games, and providing valuable feedback. You rock.

 

thank you! It looks great.

Unfortunately I can not update since I am using Unity 5.6 and now the minimum requirement is unity 2018.
Is there a chance that you include compatibility or the Miniscript update needs a newer version of unity?

 

That's not actually true, though I understand the Asset Store makes it look that way. It's only because Unity has made it quite onerous to specify what versions of Unity an asset works with.

But I assure you, there is nothing in this version of MiniScript that won't work fine on Unity 5.6.

Best,
- Joe

 

If I try to update from unity, the update button is grayed out with a message that says:
"Needs Unity upgrade to version 2018.2.15"

According to my experience with the asset store, you can only use an asset in the version of unity that says in the store not with previous versions.

Edit: In the store also says at the bottom:

Supported Unity versions
2018.2.15 or higher
Package has been submitted using Unity 2018.2.15.

 

Yes, I know it says that; Unity has made it very difficult to specify what versions an asset supports. But I didn't know that it wouldn't let you ignore that and update anyway.

I'll PM you a copy of the latest. If anybody else is in the same situation, PM me and let me know. (Or, you can also use a newer version of Unity just to download 1.3 from the Asset Store, and then copy the files into your older project.)

 

Hi,

I'm trying the new update but I need some help.

Now the parenthesis bug is fixed, but in my custom objects I have problems.
For example, in the previous version I could make a call like this

MyObject.my_method

But now I have to do:

MyObject().my_method

or I get this error:

Type Error (while attempting to look up my_method)

The strange thing is that this works:

storedObj = MyObject

sotedObj.my_method

My custom types descend from Value.
I'm missing something or can it be a bug?

Regards.

 

I'm not sure I understand what you mean about custom types descended from Value. Can you elaborate?

 

I'm extending Miniscript so that in addition to the values it has such as ValString, ValNumber, etc. I can return custom types of my own game, and those own types have their own functions.

In the same way in the MiniscriptIntrinsics.cs exists

public static ValMap ListType()

I have, for example

public static ValMap MyCustomType()

 

Taking this to PM so as to not clutter the thread... I think what you're doing there is unusual enough that it probably won't benefit others to dive into details here.

 

Some important changes (good changes!) are coming to MiniScript in the next few months, and I wanted to give everybody a heads-up about them.

First, MiniScript is going open-source! This will be just the core engine, mind you. At the same time, the Unity asset will get a Unity runtime code editor, which will not be open-source. This new version of the asset will of course be a free upgrade for all current asset customers.

Second: MiniScript is getting a C++ version! This works and feels just like the C# version. It will also be open-source, and will enable MiniScript to be easily embedded in C/C++ programs (like some other game engines you may have heard of). I'm working hard right now to find and fix any inconsistencies between the two implementations.

Third: as a result of the C++ work, there is now a command-line version of MiniScript for Mac and Linux (Windows coming this summer, if not sooner). Feel free to download it for interactive use, or to do shell scripting in a more pleasant language.

​

Also thanks to C++, we have a new way to try MiniScript in your web browser. This replaces the old "MiniScript playground" which required loading a whole Unity WebGL context; this is much lighter weight and easier for us to maintain, and easier for you to use, even on your phone or whatever.

As you can see, some of this great stuff is available now, while other bits (including the open-source repo, MiniScript user forums, etc.) are still on their way. But you guys deserved to hear about it now. Reaction from game developers and end-users (players) using MiniScript has been overwhelmingly positive, and I'm excited to see how much farther it will go when all this comes together this summer.

So give the new stuff a try, start thinking about what you might do with an in-game code editor, and as always, if you have any questions/suggestions/complaints, don't hesitate to let me know!

 

Hello @JoeStrout , that's really some great news. Regarding the in-game code editor, what features can we expect it to have ? And when do you plan to release it ? I was building an in-game code editor but progress was slow so if you are further along it would be better.

 

It will be released very soon — probably submitted to the Asset Store in a few days. It supports line numbers, syntax coloring, and all the basic selection/editing functionality you would expect. If you want to give it a try, you can go here and click the Edit button next to any reindeer.

 

@JoeStrout,

Thanks again for all of the hard work on miniscript. A quick question:

In the Integration Guide, you state the following.

"ValMap also overrides the square-brackets operator, allowing you to easily get and set values by a string index. For example, if data is a ValMap, then data["x"] is a useful (and more efficient) shorthand for data.map[new StringValue("x")]."

Does this mean that every time I use square bracket notation I am allocating memory for a new StringValue("x")? I was planning on changing values in a ValMap in a for() loop on the C# side every Update(). This would might mean several hundred new StringValue allocations per frame.

Won't this create a massive amount of garbage to be collected?

Is there a way to just mutate the existing key, rather than creating a new StringValue each time?

 

No, it does not mean that. That's what "(and more efficient)" is hinting at: when you use the square-brackets operator, it bypasses the need to create a new StringValue, and just looks up the item by the C# string key.

 

Excellent! Thanks for the clarification!