ServiceStack v5.7

.NET Core 3

Ordinarily we'd have a longer release cadence in between releases with more features, but with the recent release of .NET Core 3 we've cut this release cycle short so we can release a version of ServiceStack compatible with .NET Core 3+. Other than that the major focus on this release was #Script with many new features we're excited to share after covering the ServiceStack changes.

Sync writes disabled by default

The primary issue in supporting .NET Core 3 was accommodating its decision to disable sync Stream writes by default, in-effect disallowing most .NET Serializers from being able to write directly to the Response OutputStream. To work around this, in .NET Core 3 all sync serializers are first written to a pooled MemoryStream before being asynchronously written to the Response's Output Stream.

Essentially all Content Type Serializers (i.e. Serialization Formats) used in ServiceStack other than HTML View Engines (Razor/Markdown/JSON Report) and #Script Pages (written from ground-up to support async writes) are currently buffered in .NET Core 3. (we'll look into extending native async serialization support to our own serializers in a future release).

.NET Core 3 does allow you to turn off this restriction on a per-request basis which can be controlled by turning off buffering of sync serializers with:

SetConfig(new HostConfig {
    BufferSyncSerializers = false,
})

Which restores the existing behavior to .NET Core 3 of serializing directly to the Output Stream and marking the request with AllowSynchronousIO=true.

Internal Changes

Whilst most of ServiceStack's other internal HTTP Handlers were already being written asynchronously when ServiceStack was rewritten to be built on top of Async Handlers back in its first v4 Release, .NET Core 3's mandate highlighted our Metadata Pages needed to be refactored to use async writes as well as some existing deprecated sync Response Write APIs that needed to be marked to allow Sync. Each of the deprecated sync write APIs have async API equivalents that you can move to when ready.

Server Events Async APIs

As all Server Events existing Notification APIs were synchronous, they couldn't be refactored to use async writes, instead usage of these existing APIs have been marked with AllowSynchronousIO=true.

To perform async writes in Server Events you can use the new *Async API equivalents available to all existing sync Notification APIs:

public interface IServerEvents 
{
    Task NotifyAllAsync(string sel, object msg, CancellationToken ct=default)
    Task NotifyChannelAsync(string chan, string sel, object msg, CancellationToken ct=default)
    Task NotifySubscriptionAsync(string subId, string sel, object msg,string chan,CancellationToken ct=default)
    Task NotifyUserIdAsync(string userId, string sel, object msg, string chan, CancellationToken ct=default)
    Task NotifyUserNameAsync(string userName, string sel, object msg, string chan,CancellationToken ct=default)
    Task NotifySessionAsync(string sessionId, string sel, object msg, string chan,CancellationToken ct=default)
}

If you're utilizing any Server Event handlers, you can change them over to use new Async Handler APIs as well:

public class ServerEventsFeature : IPlugin 
{
    Func<IEventSubscription, Task> OnSubscribeAsync
    Func<IEventSubscription,Task> OnUnsubscribeAsync
    Func<IEventSubscription, IResponse, string, Task> OnPublishAsync
}

Upgrading to .NET Core 3.0

ServiceStack's .NET Core Project Templates continue to be configured to use .NET Core 2.1 LTS until .NET Core 3.1 LTS is released (scheduled for November 2019).

In the meantime you can follow these steps to manually update them to .NET Core 3:

Replace the <TargetFramework> in the Host and Test projects to target netcoreapp3.0:

- <TargetFramework>netcoreapp2.1</TargetFramework>
+ <TargetFramework>netcoreapp3.0</TargetFramework>

Remove the reference to Microsoft.AspNetCore.App in the host project:

- <PackageReference Include="Microsoft.AspNetCore.App" />

Replace IHostingEnvironment with IWebHostEnvironment and add the Microsoft.Extensions.Hosting namespace:

+ using Microsoft.Extensions.Hosting;
+ public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
- public void Configure(IApplicationBuilder app, IHostingEnvironment env)

Disable Endpoint Routing for MVC Projects using the default MVC Routing:

- services.AddMvc();
+ services.AddMvc(option => option.EnableEndpointRouting = false);

For reference here were the commits to upgrade techstacks.io, validation.web-app.io and chat.netcore.io to .NET Core 3.0:

Upgrading Docker and Travis CI to .NET Core 3.0

Projects following our Deployment to Docker AWS ECS guide requires a few more changes as travis-ci does not natively support .NET Core 3.0 yet, but it can still be installed from a snap which requires using either their xenial or bionic distro images.

A complete working .travis.yml configuration for building .NET Core 3.0 projects now looks like:

.travis.yml

dist: xenial
addons:
  snaps:
  - name: dotnet-sdk
    classic: true
    channel: latest/stable
sudo: required
language: csharp
mono: none
script:
  - sudo snap alias dotnet-sdk.dotnet dotnet
  - dotnet --version
  - chmod +x ./deploy-envs.sh
  - chmod +x ./scripts/build.sh
  - chmod +x ./scripts/deploy.sh
  - cd scripts && ./build.sh
  - if [ "$TRAVIS_BRANCH" == "master" ]; then ./deploy.sh; fi

The .NET Core 3.0 Docker Images are now being published to Microsoft's Container Registry, the recommended Dockerfile format for .NET Core 3.0 Apps now looks like:

Dockerfile

FROM mcr.microsoft.com/dotnet/core/sdk:3.0 AS build
WORKDIR /app

# copy csproj and restore as distinct layers
COPY src/*.sln .
COPY src/Chat/*.csproj ./Chat/
RUN dotnet restore

# copy everything else and build app
COPY src/Chat/. ./Chat/
WORKDIR /app/Chat
RUN dotnet publish -c Release -o out

# Build runtime image
FROM mcr.microsoft.com/dotnet/core/aspnet:3.0 AS runtime
WORKDIR /app
COPY --from=build /app/Chat/out ./
ENV ASPNETCORE_URLS http://*:5000
ENTRYPOINT ["dotnet", "Chat.dll"]

More changes in .NET Core 3.0 changes can be found in ASP.NET Core 2.2 to 3.0 Migration Guide

Troubleshooting

If ServiceStack Razor Pages have issues resolving views after upgrading to .NET Core 3.0, disable pre-compiled views:

<MvcRazorCompileOnPublish>false</MvcRazorCompileOnPublish>

See this Customer Forums Post for more info.

Upgrade to Fluent Validation v8.5

ServiceStack's interned version of Fluent Validation has been updated to its latest v8.5 Release.

Interestingly this is the first time a Sharp Script was used to partially automate the update. The productivity of the real-time feedback in Sharp Scripts crosses the threshold for it being faster to automate in-frequent one-off tasks like this than performing the update manually.

Service Plugin APIs

The new Plugin APIs in the Service base class lets you avoid singleton access to HostContext.GetPlugin<T>() within your Service implementation:

T GetPlugin<T>() - returns null when plugin not registered
T AssertPlugin<T>() - throw if plugin not registered

AuthFeature

`/authenticate` alias routes removed by default

Historically ServiceStack included a more formal /authenticate alias that could be used instead of the shorter /auth route for accessing its Authentication services. But only the /auth route was documented and the longer alias didn't see much usage except when it appeared in the Services metadata which presented a potential source of confusion as it included were duplicated routes for Auth.

We've decided to remove the /authenticate alias by default, you can use AddAuthenticateAliasRoutes() to re-add it when registering the AuthFeature plugin, e.g:

Plugins.Add(new AuthFeature(...) {
    ...
}.AddAuthenticateAliasRoutes());

External Redirects used in the ?continue params of /auth requests are disabled by default, they can be re-enabled with:

new AuthFeature(...) {
    ValidateRedirectLinks = AuthFeature.AllowAllRedirects 
}

OrmLite

New OrmLite Packages have been published that makes use of Microsoft's alternative Microsoft.Data.SQLite and Microsoft.Data.SqlClient ADO.NET Data providers:

ServiceStack.OrmLite.Sqlite.Data - Sqlite provider that uses Microsoft.Data.SQLite
ServiceStack.OrmLite.SqlServer.Data - SqlServer provider that uses Microsoft.Data.SqlClient
ServiceStack.OrmLite.SqlServer.Data.Core - Use Microsoft.Data.SqlClient on ASP.NET Core on .NET Framework

They're source-code compatible with OrmLite's existing OrmLite.Sqlite and OrmLite.SqlServer packages where App's will be able to easily switch to these new packages by adding the .Data suffix to their existing OrmLite NuGet package references.

GetSchemaTable

ADO.NET's IDataReader.GetSchemaTable() for retrieving schema info on a query, can now be accessed from OrmLite with:

DataTable schema = db.GetSchemaTable("SELECT * from Table");

Although as this returns an unstructured dump of attributes in a DataTable we've also provided APIs to return the results a typed ColumnSchema POCO:

ColumnSchema[] schema = db.GetTableColumns<Table>();
ColumnSchema[] schema = db.GetTableColumns(typeof(Table));
ColumnSchema[] schema = db.GetTableColumns("SELECT * from Table");

async equivalents available for all new APIs

DB Script Methods

As this Table Schema information is useful when accessing databases in #Script with DB Scripts, this information is also available from:

'Table'.dbColumnNames()
'Table'.dbColumns() |> map => `${it.ColumnName} ${it.DataTypeName}` |> joinln
'select * from Table'.dbDesc() |> map => `${it.ColumnName} ${it.DataTypeName}` |> joinln

ServiceStack.Text

The MemoryProvider abstraction is used to take advantage of .NET Core's more performant and low allocation APIs whilst still being able to support .NET Framework from the same code-base. New APIs added in this release include:

class MemoryProvider
{
    abstract Task WriteAsync(Stream stream, ReadOnlySpan<char> value, CancellationToken token=default);
    public abstract void Write(Stream stream, ReadOnlyMemory<char> value);
    public abstract void Write(Stream stream, ReadOnlyMemory<byte> value);
}

Auto Mapping

Auto Mapping now makes usage of implicit casts defined on the target Type as well, e.g:

var xname = "Text".ConvertTo<XName>(); //uses implicit string cast in XName

New Env.IsNetCore3 can be used to detect if running on .NET Core 3.x
min() and max() APIs added to DynamicNumber

#Script

We've seen great reception of Gist Desktop Apps from the last v5.6 release with a nice shoutout from Jon Galloway in the ASP.NET Community August Stand up, a featured blog post from Scott Hanselman as well features in Hacker News and Reddit.

We've since further enhanced #Script capabilities in this release making it more functional then ever, broadening its appeal in its growing list of use-cases.

1st class #Script Code and Lisp Language support!

In addition to enhanced capabilities, it also gains support for multiple languages with #Script code now implemented as a first-class language as well entirely new support for Lisp! - one of the smallest and most powerful languages in existence whose dynamism and extensibility makes it particularly well suited for a range of explanatory programming tasks that can now be harnessed in .NET Core and .NET Framework Apps.

You can interactively explore and compare each of #Script languages in C#'s 101 LINQ Examples:

template code lisp

Transition to JS Pipeline Operator Syntax

First we want to announce our intention to move to use JavaScript's proposed |> Pipeline Operator syntax that's now moved to stage 1 proposal where it's become clear that they've chosen to use F#'s pipe forward |> operator for its syntax. It's currently only at Stage 1 so could still be years before it's adopted in browsers as there's still open questions about its exact semantics, but a version of it is currently available as a babel plugin and in Firefox under an experimental --enable-pipeline-operator flag.

We'd also like to transition to use the new |> syntax for future benefits in improved familiarity for JS developers as well as the source-code compatibility benefits from code portability to compatibility with JS tools, language services and syntax highlighters.

As the existing | operator has been fundamental in existing #Script code bases we want to transition to the new syntax with the least disruption possible by supporting both syntaxes in parallel over multiple releases so existing code-bases can transition at their own pace, which just involves replacing existing usages of | to use |>, e.g:

{‎{ 'shout' | upper | substring(2) | padRight(6, '_') | repeat(3) }‎}

//to

{‎{ 'shout' |> upper |> substring(2) |> padRight(6, '_') |> repeat(3) }‎}

When ready you can enforce that only the new |> syntax is used by disallowing | usage with:

ScriptConfig.AllowUnixPipeSyntax = false;

F#'s |> pipe forward operator has now been implemented in multiple languages which works as expected when pipe forwarding to a function with a single argument:

a |> f = f(a)

But the semantics of how it's applied when forwarding to functions with multiple arguments varies such that a |> f(b) desugars to:

f(a, b)  // Elixir first argument
f(b, a)  // F# last argument
f(b)(a)  // JS proposal

#Script like Elixir passes it as the first argument which in our opinion is the most intuitive behavior that better supports calling overloaded functions with different parameter counts.

Scripting .NET!

By default the only functionality and objects #Script has access to is what's pre-configured within a new ScriptContext sandbox which has access to Default Scripts, HTML Scripts and default Script Blocks. #Script can't call methods on instances or have any other way to invoke a method unless it's explicitly registered.

To enable additional functionality, the ScriptContext that executes your script can be extended with additional:

var context = new ScriptContext {
    Args = { ... },              // Global Arguments available to all Scripts, Pages, Partials, etc
    ScriptMethods = { ... },     // Additional Methods
    ScriptBlocks = { ... },      // Additional Script Blocks 
    FilterTransformers = { .. }, // Additional Stream Transformers
    PageFormats = { ... },       // Additional Text Document Formats
    Plugins = { ... },           // Encapsulated Features e.g. Markdown, Protected or ServiceStack Features

    ScanTypes = { ... },         // Auto register Methods, Blocks and Code Page Types
    ScanAssemblies = { ... },    // Auto register all Methods, Blocks and Code Page Types in Assembly
}.Init();

Although being able to Script .NET Types directly gives your Scripts greater capabilities and opens it up to a lot more use-cases that's especially useful in predominantly #Script-heavy contexts like Sharp Apps and Shell Scripts giving them maximum power that would otherwise require the usage of Plugins.

We can visualize the different scriptability options from the diagram below where by default scripts are limited to functionality defined within their ScriptContext, whether to limit access to specific Types and Assemblies or whether to lift the escape hatch and allow scripting of any .NET Types.

The new .NET Scripting support is only available to Script's executed within trusted contexts that are registered with Protected Scripts. The different ways to allow scripting of .NET Types include:

Script Assemblies and Types

Using ScriptTypes to limit scriptability to only specific Types:

var context = new ScriptContext {
    ScriptMethods = {
        new ProtectedScripts()
    },
    ScriptTypes = {
        typeof(MyType),
        typeof(MyType2),
    }
}.Init();

Or you can use ScriptAssemblies to allow scripting of all Types within an Assembly:

var context = new ScriptContext {
    ScriptMethods = {
        new ProtectedScripts()
    },
    ScriptAssemblies = {
        typeof(MyType).Assembly,
    }
}.Init();

AllowScriptingOfAllTypes

To give your Scripts maximum accessibility where they're able to pierce the well-defined ScriptContext sandbox, you can set AllowScriptingOfAllTypes to allow scripting of all .NET Types available in loaded assemblies:

var context = new ScriptContext {
    ScriptMethods = {
        new ProtectedScripts()
    },
    AllowScriptingOfAllTypes = true,
    ScriptNamespaces = {
        typeof(MyType).Namespace,
    }
}.Init();

ScriptNamespaces is used to include additional Lookup namespaces for resolving Types akin to C# using statements.

Using AllowScriptingOfAllTypes also allows access to both public and non-public Types.

Scripting .NET APIs

The following Protected Scripts are all that's needed to create new instances, call methods and populate instances of .NET Types, including generic types and generic methods.

// Resolve Types
Type typeof(string typeName);

// Call Methods
object call(object instance, string name);
object call(object instance, string name, List<object> args);
Delegate Function(string qualifiedMethodName);                    // alias F(string)

// Create Instances
object new(string typeName);
object new(string typeName, List<object> constructorArgs);
object createInstance(Type type);
object createInstance(Type type, List<object> constructorArgs);
ObjectActivator Constructor(string qualifiedConstructorName);     // alias C(string)

// Populate Instance
object set(object instance, Dictionary<string, object> args);

Note: only a Type's public members can be accessed from #Script

Type Resolution

If you've registered Types using either ScriptTypes or ScriptAssemblies than you'll be able to reference the Type using just the Type Name, unless multiple Types of the same name are registered in which case the typeof() will return the first Type registered, all other subsequent Types with the same Name will need to be referenced with their Full Name.

typeof('MyType')
typeof('My.Namespace.MyType')

When AllowScriptingOfAllTypes=true is enabled, you can use ScriptNamespaces to add Lookup Namespaces for resolving Types, which for #Script Pages, Sharp Apps and Sharp Scripts are pre-configured with:

var context = new ScriptContext {
    //...
    ScriptNamespaces = {
        "System",
        "System.Collections.Generic",
        "ServiceStack",
    }
}.Init();

All other Types (other than .NET built-in types) not registered in ScriptTypes, ScriptAssemblies or have their namespace defined in ScriptNamespaces will need to be referenced using their Full Type Name. This same Type resolution applies for all references of Types in #Script.

Examples Configuration

The examples below assumes a ScriptContext configured with:

var context = new ScriptContext {
    ScriptMethods = { new ProtectedScripts() },
    AllowScriptingOfAllTypes = true,
    ScriptNamespaces = {
        "System",
        "System.Collections.Generic",
    },
    ScriptTypes = {
        typeof(Ints),
        typeof(Adder),
        typeof(StaticLog),
        typeof(InstanceLog),
        typeof(GenericStaticLog<>),
    },
}.Init();

With the types for the above classes defined in ScriptTypes.cs. This is the definition of the Adder class that's referenced frequently in the examples below:

public class Adder
{
    public string String { get; set; }
    public double Double { get; set; }

    public Adder(string str) => String = str;
    public Adder(double num) => Double = num;

    public string Add(string str) => String += str;
    public double Add(double num) => Double += num;

    public override string ToString() => String != null ? $"string: {String}" : $"double: {Double}";
}

Creating Instances

There are 3 different APIs for creating instances of Types:

new - create instances from Type name with specified List of arguments
createInstance - create instance of Type with specified List of arguments
Constructor - create a Constructor delegate that can create instances via method invocation

Built-in .NET Types and Types in ScriptTypes, ScriptAssemblies or ScriptNamespaces can be created using their Type Name, including generic Types:

'int'.new()
'DateTime'.new()
'Dictionary<string,DateTime>'.new()

Otherwise new instances of Types can be created using their full Type Name:

'System.Int32'.new()
'System.Text.StringBuilder'.new()

A list of arguments can be passed to the new method to call the constructor with the specified arguments:

'Ints'.new([1,2])
'Adder'.new([1.0])
'KeyValuePair<string,int>'.new(['A',1])

Constructor Resolution

#Script will use the constructor that matches the same number of specified arguments, when needed it uses ServiceStack's Auto Mapping to convert instances when their Types don't match, e.g:

'Ints'.new([1.0,2.0])
'KeyValuePair<char,double>'.new(['A',1])

However if there are multiple constructors with the same number of arguments, it will only use the constructor where all its argument Types match with the supplied arguments. Attempting to create an instance of the Adder class which only has constructors for string or double will fail with an Ambiguous Match Exception when trying to create it with an int:

'Adder'.new([1])  // FAIL: Ambiguous Constructor

In this case you'll need to convert the arguments so its Types matches one of the available constructors:

'Adder'.new([1.0])
'Adder'.new([intArg.toDouble()])
'Adder'.new(['A'])
'Adder'.new([`${instance}`]) // or 'Adder'.new([instance.toString()])

Constructor function

Alternatively you can use the Constructor method to specify the constructor you want by specifying the argument types of the constructor you want to use, which will return a delegate that lets you call a method to create instances using that Type's constructor:

Constructor('Adder(double)') |> to => doubleAdder
Constructor('Adder(string)') |> to => stringAdder

In this case you will be able to create instances of Adder using an int argument as the built-in automapping will convert it to the Argument Type of the Constructor you've chosen:

doubleAdder(1)
stringAdder(1)

// equivalent to:
Constructor('Adder(double)')(1)
Constructor('Adder(string)')(1)

As the Constructor Function returns a delegate you will be able to invoke it like a normal method where it can also be invoked as an extension method or inside a filter expression:

Constructor('Uri(string)') |> to => url

url('http://example.org')
'http://example.org'.url()
'http://example.org' |> url

// equivalent to:
'Uri'.new(['http://example.org'])
Constructor('Uri(string)')('http://example.org')

C() alias

To reduce syntax noise when needing to create a lot of constructors you can use the much shorter alias C instead of Constructor:

C('Uri(string)') |> to => url
C('Adder(double)')(1)

createInstance

The createInstance is like new except it's used to create instances from a Type instead of its string Type Name:

typeof('Ints').createInstance([1,2])
typeof('Adder').createInstance([1.0])
typeof('KeyValuePair<string,int>').createInstance(['A',1])

set

Once you've created instance you can further populate it using the set method which will let you populate public properties with a JS Object literal, performing any auto-mapping conversions as needed:

'Ints'.new([1,2]).set({ C:3, D:4.0 })
Constructor('Ints(int,int)')(1,2).set({ C:3, D:4.0 })

As set returns the instance, it can be used within a chained expression:

instance.set({ C:3 }).set({ D:4.0 }).call('GetTotal')

Calling Methods

Use the call and Function APIs to call methods on .NET Types:

call - invoke a method on an instance
Function - create a Function delegate that can invoke methods via normal method invocation

call

In its most simplest form you can invoke an instance method that doesn't have any arguments using just its name:

'Ints'.new([1,2]) |> to => ints
ints.call('GetMethod')

Any arguments can be specified in an arguments list:

'Adder'.new([1.0,2.0]) |> to => adder3
adder3.call('Add',[3.0]) //= 6.0

Method Resolution

The same Resolution rules in Constructor Resolution also applies when calling methods where any ambiguous methods needs to be called with arguments containing the exact types (as above), or you can specify the argument types of the method you want to call, in which case it will let you use the built-in Auto Mapping to call a method expecting a double with an int argument:

adder3.call('Add(double)',[3])

Generic Methods

You can call generic methods by specifying the Generic Type in the method name:

'Ints'.new([1,2]).call('GenericMethod<string>',['A'])

call only invokes instance methods, to call static methods you'll need to use Function.

Function

Function is a universal accessor for .NET Types where it can create a cached delegate to access Instance, Static and Generic Static Types - Including Nested Types (aka Inner Classes), Instance, Static and Generic Methods of those Types as well as their Instance and Static Properties, Fields and Constants.

As a simple example we'll use Function to create a delegate to call .NET's System.Console.WriteLine(string) static method:

Function('Console.WriteLine(string)') |> to => writeln

Which lets you call it like a regular Script method:

writeln('A')
'A'.writeln()
Function('Console.WriteLine(string)')('A')

All Examples below uses classes defined in ScriptTypes.cs.

Instance Methods

Function create delegates that lets you genericize the different types of method invocations in .NET, including instance methods, generic methods and void Action methods on an instance:

'InstanceLog'.new(['A']) |> to => o
Function('InstanceLog.Log') |> to => log              // instance void method
Function('InstanceLog.AllLogs') |> to => allLogs      // instance method
Function('InstanceLog.Log<int>') |> to => genericLog  // instance generic method

o.log('B')
log(o,'C')
o.genericLog(1)
o | genericLog(2)    
o.allLogs() |> to => snapshotLogs

Static Type Methods

As well as calling static methods and static void Action methods on a static Type:

Function('StaticLog.Clear')()
Function('StaticLog.Log') |> to => log                // static void method
Function('StaticLog.AllLogs') |> to => allLogs        // static method
Function('StaticLog.Log<int>') |> to => genericLog    // static generic method

log('A')
'B'.log()
genericLog('C')
allLogs() |> to => snapshotLogs

Generic Static Type Methods

Including calling generic static methods on a generic static Type:

Function('GenericStaticLog<string>.Clear()')()
Function('GenericStaticLog<string>.Log(string)') |> to => log      // generic type static void method
Function('GenericStaticLog<string>.AllLogs') |> to => allLogs      // generic type static method
Function('GenericStaticLog<string>.Log<int>') |> to => genericLog  // generic type generic static method

log('A')
'B'.log()
genericLog('C')
allLogs() |> to => snapshotLogs

F() alias

You can use the shorter F() alias to reduce syntax noise when writing #Script that heavily interops directly with .NET Classes.

Instance and Static Properties, Fields and Constants

In addition to being able to create Delegates that genericize access to .NET Methods, it can also be used to create a delegate for accessing Instance and Static Properties, Fields and Constants including members of Inner Classes, e.g:

Each of the members of the following Type definition:

public class StaticLog
{
    public static string Prop { get; } = "StaticLog.Prop";
    public static string Field = "StaticLog.Field";
    public const string Const = "StaticLog.Const";

    public string InstanceProp { get; } = "StaticLog.InstanceProp";
    public string InstanceField = "StaticLog.InstanceField";

    public class Inner1
    {
        public static string Prop1 { get; } = "StaticLog.Inner1.Prop1";
        public static string Field1 = "StaticLog.Inner1.Field1";
        public const string Const1 = "StaticLog.Inner1.Const1";

        public string InstanceProp1 { get; } = "StaticLog.Inner1.InstanceProp1";
        public string InstanceField1 = "StaticLog.Inner1.InstanceField1";

        public static class Inner2
        {
            public static string Prop2 { get; } = "StaticLog.Inner1.Inner2.Prop2";
            public static string Field2 = "StaticLog.Inner1.Inner2.Field2";
            public const string Const2 = "StaticLog.Inner1.Inner2.Const2";
        }
    }
}

Can be accessed the same way, where you can use Function to create a zero-argument delegate for static members that can be immediately invoked, or a 1 argument Delegate for instance members.

Examples below uses Function's shorter F() alias:

F('StaticLog.Prop')()
F('StaticLog.Field')()
F('StaticLog.Const')()

F('StaticLog.Inner1.Prop1')()
F('StaticLog.Inner1.Field1')()
F('StaticLog.Inner1.Const1')()

F('StaticLog.Inner1.Inner2.Prop2')()
F('StaticLog.Inner1.Inner2.Field2')()
F('StaticLog.Inner1.Inner2.Const2')()

'StaticLog'.new() |> to => o
F('StaticLog.InstanceProp')(o)
F('StaticLog.InstanceField')(o)

'StaticLog.Inner1'.new() |> to => o
F('StaticLog.Inner1.InstanceProp1')(o)
F('StaticLog.Inner1.InstanceField1')(o)

#Script Code

The initial support for code statements was implemented using a simple pre-processor that wrapped each line within a template expression - a technique inspired by CoffeeScript who used input source code transformation to reduce implementation burden.

However the naivety of this approach showed itself when implementing smarter Script Blocks whose body you'd want precise control over depending on the type of functionality it provides and the context from where it's used.

For the new CSV and keyvalues Script Blocks used in Live Documents you'll always want to ensure the body is captured in unstructured free-text, e.g:

    {‎{#keyvalues monthlyRevenues ':'}‎}
        Salary:         4000
        App Royalties:  200
    {‎{/keyvalues}‎}

Likewise when used in code statements:

    #keyvalues monthlyExpenses
        Rent            1000
        Internet        50
        Mobile          50
        Food            400
        Misc            200
    /keyvalues

But for blocks like capture which captures dynamically generated output, it's preferred the body be expressed using templates:

{‎{#capture out}‎}
    <ul>
    {‎{#each range(3)}‎}
        <li>{‎{it}‎}</li>
    {‎{/each}‎}
    </ul>
{‎{/capture}‎}

Even when used from within code statements:

#capture out
    <ul>
    {‎{#each range(3)}‎}
        <li>{‎{it}‎}</li>
    {‎{/each}‎}
    </ul>
/capture

Whilst the new function blocks body should always be defined using code statement block:

#function fib(num)
    #if num <= 1
        return(num)
    /if
    return (fib(num-1) + fib(num-2))
/function

Likewise when defined in "template" mode:

{‎{#function fib(num) }‎}
    #if num <= 1
        return(num)
    /if
    return (fib(num-1) + fib(num-2))
{‎{/function}‎}

None of this was possible with our previous naive implementation. In Razor terms, this is akin to trying to support using Tag libraries from within a C# Statement blocks:

@{
    <environment names="Staging,Production">
        <strong>HostingEnvironment.EnvironmentName is Staging or Production</strong>
    </environment>    
}

The easiest solution would be to not support it and only allow script blocks when in "Template Mode".

But as Script Blocks are a powerful tool for defining nearly any kind of DSL and code statement blocks being preferred for all other use-cases that doesn't involve generating text output, we decided to implement it properly, with code statement blocks being implemented as a first-class language and script blocks being able to specify how their body should be parsed.

As a first-class language we can also offer the same language services as #Script templates:

Executing #Script Code in .NET

To capture rendered output from code blocks you can use the new RenderCode* APIs:

// render code statements
var output = context.RenderCode("now |> dateFormat('HH:mm:ss')"); 

// async
var output = await context.RenderCodeAsync("now |> dateFormat('HH:mm:ss')");

These APIs match the high-level APIs for rendering normal #Script:

var output = context.RenderScript("{‎{ now |> dateFormat('HH:mm:ss') }‎}"); 
var output = await context.RenderScriptAsync("{‎{ now |> dateFormat('HH:mm:ss') }‎}");

Finer grained control

The high-level APIs above wraps the finer-grained functionality below which works by rendering a SharpPage configured with the code language in a PageResult that all languages use:

var context = new ScriptContext().Init();
var dynamicPage = context.CodeSharpPage("now |> dateFormat('HH:mm:ss')");           // render code
//var dynamicPage = context.SharpScriptPage("{‎{ now |> dateFormat('HH:mm:ss') }‎}"); // render #Script
var output = new PageResult(dynamicPage).RenderScript();

//async
var output = await new PageResult(dynamicPage).RenderScriptAsync();

To instead capture the return value of a script block you can use the new EvaluateCode APIs, e.g:

var result = context.EvaluateCode("return (1 + 1)"); //= 2

The generic overloads below utilizes ServiceStack's Auto Mapping utils to convert the return value into your preferred type, e.g:

double result = context.EvaluateCode<double>("return (1 + 1)"); //= 2.0
string result = context.EvaluateCode<string>("return (1 + 1)"); //= "2"

Which can also be used for more powerful conversions like converting an Object Dictionary into your preferred POCO:

var result = context.EvaluateCode<Customer>("`select * from customer where id=@id` |> dbSingle({id}) |>return"
    , new ObjectDictionary {
        ["id"] = 1
    });

Code Scripts

The same functionality in Sharp Scripts is also available in #Script Code, except instead of using the *.ss file extension for executing #Script you'd use the *.sc file extension which will allow you to use the web and app dotnet tools to watch or run code scripts:

$ web run code.sc
$ web watch code.sc

Watched code scripts

Here's a quick demo showcasing the same functionality in Sharp Scripts is also available in *.sc scripts which provides instant feedback whilst you develop in real-time:

YouTube: youtu.be/TQPOZ0kVpw4

When you've re-designed your library to support multiple languages, the next one becomes much easier to add :) But still being mindful of library size, we'd only want to include support for small and powerful languages - and there's few with a better power to weight ratio than Lisp!

Introducing #Script Lisp!

#Script is itself designed as a small, expressive and wrist-friendly dynamic scripting language that for maximum familiarity is modelled after the world's most popular and ubiquitous scripting Language, JavaScript. Its minimal syntax was inspired by other small but powerful languages which heavily utilizes functions instead of adopting a larger language grammar defining different bespoke syntax for language constructs.

Small Languages like Smalltalk, despite being one of the most influential languages in history, is famous for its minimal syntax that fits on a post card. A language with arguably better power to size ratio is Lisp which the inventor of Smalltalk, Alan Kay has credited it as being the greatest single programming language ever designed after realizing:

“the half page of code on the bottom of page 13… was Lisp in itself. These were “Maxwell’s Equations of Software!”

Lisp's unprecedented elegance and simplicity spawned a myriad of dialects, some noteworthy implementations illustrating the beauty of its small size and expressive power is lispy by by Peter Norvig (Director of Google Research) that implements a Lisp interpreter in just 117 lines of Python 3 code (inc. a REPL).

Another compact dialect is Zick Standard Lisp which @zick has implemented in 42 different languages including a recursive Lisp evaluator in Lisp implemented in only 66 lines of code.

A more complete Lisp implementation in C# is the elegant Nukata Lisp by SUZUKI Hisao which is a Common Lisp-like Lisp-1 dialect with tail call optimization and partially hygienic macros, although has some notable limitations including a small standard library, only uses the double numeric type and doesn't contain .NET Scripting support.

Script Lisp Overview

ScriptLisp is an enhanced version of Nukata Lisp with a number of new features that reuses #Script existing scripting capabilities to provide seamless integration with both the rest of #Script (see Language Blocks an Expressions) and .NET including Scripting of .NET Types, support for all .NET numeric types and access to its comprehensive library of over 1000+ Script Methods - optimally designed for accessing .NET functionality from a dynamic language.

To improve compatibility with existing Common Lisp source code it also implements most of the Simplified Common Lisp Reference as well as all missing functions required to implement C# LINQ 101 Examples in Lisp:

Lisp LINQ Examples

To improve readability and familiarity it also adopts a number of Clojure syntax for defining a data list and map literals, anonymous functions, syntax in Java Interop for .NET Interop, keyword syntax for indexing collections and accessing index accessors and Clojure's popular shorter aliases for fn, def, defn - improving source-code compatibility with Clojure.

Lisp REPL

In addition to being a 1st class language option in #Script, Lisp's dynamism and extensibility makes it particularly well suited for explanatory programming whose access via a REPL is now built into the latest x and app dotnet tools which can be quickly installed in any Windows, macOS or Linux OS (with .NET Core) with:

$ dotnet tool install -g web

Or if you have a previous version installed, update to the latest version with:

$ dotnet tool update -g web

Where you'll then be able to bring up an instant Lisp REPL with:

$ web lisp

The quick demo below shows the kind of exploratory programming available where you can query the scriptMethods available, query an objects props, query the Lisp interpreter's global symbols table containing all its global state including all defined lisp functions, macros and variables:

YouTube: youtu.be/RR7yk6ReNnQ

Annotated REPL Walk through

Here's an annotated version of the demo below which explains what each of the different expressions is doing.

Just like Sharp Scripts and Sharp Apps the Lisp REPL runs within the #Script Pages ScriptContext sandbox that when run from a Sharp App folder, starts a .NET Core App Server that simulates a fully configured .NET Core App. In this case it's running in the redis Sharp App directory where it was able to access its static web assets as well as its redis-server connection configured in its app.settings.

; quick lisp test!
(+ 1 2 3)

; List of ScriptMethodInfo that the ScriptContext running this Lisp Interpreter has access to
scriptMethods

; first script method
(:0 scriptMethods)

; show public properties of ScriptMethodInfo 
(props (:0 scriptMethods))

; show 1 property per line
(joinln (props (:0 scriptMethods)))

; show both Property Type and Name
(joinln (propTypes (:0 scriptMethods)))

; view the Names of all avaialble script methods
(joinln (map .Name scriptMethods))

; view all script methods starting with 'a'
(globln "a*" (map .Name scriptMethods))

; view all script methods starting with 'env'
(globln "env*" (map .Name scriptMethods))

; print environment info about this machine seperated by spaces
(printlns envOSVersion envMachineName envFrameworkDescription envLogicalDrives)

; expand logical drives
(printlns envOSVersion envMachineName envFrameworkDescription "- drives:" (join envLogicalDrives " "))

; view all current global symbols defined in this Lisp interpreter
symbols

; view all symbols starting with 'c'
(globln "c*" symbols)

; see how many symbols are defined in this interpreter
(count symbols)

; see how many script methods there are available
(count scriptMethods)

; view the method signature for all script methods starting with 'all'
(globln "all*" (map .Signature scriptMethods))

; count all files accessible from the configured ScriptContext
(count allFiles)

; view the public properties of the first IVirtualFile
(props (:0 allFiles))

; display the VirtualPath of all available files
(joinln (map .VirtualPath allFiles))

; display the method signature for all script methods starting with 'findFiles'
(globln "findFiles*" (map .Signature scriptMethods))

; see how many .html files are available to this App
(count (findFiles "*.html"))

; see how many .js files are available to this App
(count (findFiles "*.js"))

; show the VirtualPath of all .html files
(joinln (map .VirtualPath (findFiles "*.html")))

; view the VirtualPath's of the 1st and 2nd .html files
(:0 (map .VirtualPath (findFiles "*.html")))
(:1 (map .VirtualPath (findFiles "*.html")))

; view the text file contents of the 1st and 2nd .html files
(fileTextContents (:0 (map .VirtualPath (findFiles "*.html"))))
(fileTextContents (:1 (map .VirtualPath (findFiles "*.html"))))

; display the method signatures of all script methods starting with 'redis'
(globln "redis*" (map .Signature scriptMethods))

; search for all Redis Keys starting with 'urn:' in the redis-server instances this App is configured with
(redisSearchKeys "urn:*")

; display the first redis search entry
(:0 (redisSearchKeys "urn:*"))

; display the key names of all redis keys starting with 'urn:'
(joinln (map :id (redisSearchKeys "urn:*")))

; find out the redis-server data type of the 'urn:tags' key
(redisCall "TYPE urn:tags")

; view all tags in the 'urn:tags' sorted set
(redisCall "ZRANGE urn:tags 0 -1")

; view the string contents of the 'urn:question:1' key
(redisCall "GET urn:question:1")

; parse the json contents of question 1 and display its tag names
(:Tags (parseJson (redisCall "GET urn:question:1")))

; extract the 2nd tag of question 1
(:1 (:Tags (parseJson (redisCall "GET urn:question:1"))))

; clear the Console screen
clear

; exit the Lisp REPL
quit

Enable features and access resources with app.settings

You can configure the Lisp REPL with any of the resources and features that Sharp Apps and Gist Desktop Apps have access to, by creating a plain text app.settings file with all the features and resources you want the Lisp REPL to have access to, e.g. this Pure Cloud App app.settings allows the Lisp REPL to use Database Scripts against a AWS PostgreSQL RDS server and query remote S3 Virtual Files using Virtual File System APIs:

# Note: values prefixed with '$' are resolved from Environment Variables
name AWS S3 PostgreSQL Web App
db postgres
db.connection $AWS_RDS_POSTGRES
files s3
files.config {AccessKey:$AWS_S3_ACCESS_KEY,SecretKey:$AWS_S3_SECRET_KEY,Region:us-east-1,Bucket:rockwind}

See the plugins app.settings for examples of how to load and configure ServiceStack Plugins.

Lisp REPL TCP Server

In addition to launching a Lisp REPL from the Console above, you can also open a Lisp REPL into any ServiceStack App configured with the LispReplTcpServer ServiceStack plugin. This effectively opens a "programmable gateway" into any ServiceStack App where it's able to perform live queries, access IOC dependencies, invoke internal Server functions and query the state of a running Server which like the Debug Inspector can provide invaluable insight when diagnosing issues on a remote server.

To see it in action we'll enable it one of our production Apps techstacks.io which as it's a Vuetify SPA App is only configured with an empty SharpPagesFeature as it doesn't use any server-side scripting features.

We'll enable it in DebugMode where we can enable by setting DebugMode in our App's appsettings.Production.json which will launch a TCP Socket Server which by default is configured to listen to the loopback IP on port 5005.

if (Config.DebugMode)
{
    Plugins.Add(new LispReplTcpServer {
        ScriptMethods = {
            new DbScripts()
        },
        ScriptNamespaces = {
            nameof(TechStacks),
            $"{nameof(TechStacks)}.{nameof(ServiceInterface)}",
            $"{nameof(TechStacks)}.{nameof(ServiceModel)}",
        },
    });
}

ScriptNamespaces behaves like C#'s using Namespace; statement letting you reference Types by Name instead of its fully-qualified Namespace.

Whilst you can now connect to it with basic telnet, it's a much nicer experience to use it with the rlwrap readline wrap utility which provides an enhanced experience with line editing, persistent history and completion.

$ sudo apt-get install rlwrap

Then you can open a TCP Connection to connect to a new Lisp REPL with:

$ rlwrap telnet localhost 5005

Where you now have full scriptability of the running server as allowed by #Script Pages SharpPagesFeature which allows scripting of all .NET Types by default.

TechStacks TCP Lisp REPL Demo

In this demo we'll explore some of the possibilities of scripting the live techstacks.io Server where we can resolve IOC dependencies to send out tweets using its registered ITwitterUpdates dependency, view the source and load a remote parse-rss lisp function into the new Lisp interpreter attached to the TCP connection, use it to parse Hacker News RSS Feed into a .NET Collection where it can be more easily queried using its built-in functions which is used to construct an email body with HN's current Top 5 links.

It then uses DB Scripts to explore its configured AWS RDS PostgreSQL RDBMS, listing its DB tables and viewing its column names and definitions before retrieving the Email addresses of all Admin users, sending them each an email with HN's Top 5 Links by publishing 5x SendEmail Request DTOs using the publishMessage ServiceStack Script to where they're processed in the background by its configured MQ Server that uses it to execute the SendEmail ServiceStack Service where it uses its configured AWS SES SMTP Server to finally send out the Emails:

YouTube: youtu.be/HO523cFkDfk

Password Protection

Since TCP Server effectively opens your remote Server up to being scripted you'll want to ensure the TCP Server is only accessible within a trusted network, effectively treating it the same as Redis Security Model.

A secure approach would be to leave the default of only binding to IPAddress.Loopback so only trusted users with SSH access will be able to access it, which they'll still be able to access remotely via Local PC > ssh > telnet 127.0.0.1 5005.

Just like Redis AUTH you can also add password protection for an additional layer of Security:

Plugins.Add(new LispReplTcpServer {
    RequireAuthSecret = true,
    ...
});

Which will only allow access to users with the configured AuthSecret:

SetConfig(new HostConfig { 
    AdminAuthSecret = "secretz" 
});

Annotated Lisp TCP REPL Transcript

; resolve `ITwitterUpdates` IOC dependency and assign it to `twitter`
(def twitter (resolve "ITwitterUpdates"))

; view its concrete Type Name
(typeName twitter)

; view its method names 
(joinln (methods twitter))

; view its method signatures 
(joinln (methodTypes twitter))

; use it to send tweet from its @webstacks account
(.Tweet twitter "Who's using #Script Lisp? https://sharpscript.net/lisp")

; view all available scripts in #Script Lisp Library Index gist.github.com/3624b0373904cfb2fc7bb3c2cb9dc1a3
(gistindex)

; view the source code of the `parse-rss` library
(load-src "index:parse-rss")

; assign the XML contents of HN's RSS feed to `xml`
(def xml (urlContents "https://news.ycombinator.com/rss"))

; preview its first 1000 chars
(subString xml 0 1000)

; use `parse-rss` to parse the RSS feed into a .NET Collection and assign it to `rss`
(def rss (parse-rss xml))

; view the `title`, `description` and the first `item` in the RSS feed:
(:title rss)
(:description rss)
(:0 (:items rss))

; view the links of all RSS feed items
(joinln (map :link (:items rss)))

; view the links and titles of the top 5 news items
(joinln (map :link (take 5 (:items rss))))
(joinln (map :title (take 5 (:items rss))))

; construct a plain-text numbered list of the top 5 HN Links and assign it to `body`
(joinln (map-index #(str %2 (:title %1)) (take 5 (:items rss))))
(joinln (map-index #(str (padLeft (1+ %2) 2) ". " (:title %1)) (take 5 (:items rss))))
(def body (joinln 
    (map-index #(str (padLeft (1+ %2) 2) ". " (:title %1) "\n" (:link %1) "\n") (take 5 (:items rss)))))

; view all TechStacks PostgreSQL AWS RDS tables
(dbTableNames)
(joinln dbTableNames)

; view the column names and definitions of the `technology` table
(joinln (dbColumnNames "technology"))
(joinln (dbColumns "technology"))

; search for all `user` tables
(globln "*user*" (dbTableNames))

; view how many Admin Users with Emails there are
(dbScalar "select count(email) from custom_user_auth where roles like '%Admin%'")

; assign the Admin Users email to the `emails` list
(def emails (map :email (dbSelect "select email from custom_user_auth where roles like '%Admin%'")))

; search for all `operation` script methods
(globln "*operation*" scriptMethods)

; search for all `email` Request DTOs
(globln "*email*" metaAllOperationNames)

; view the properties available on the `SendEmail` Request DTO
(props (SendEmail.))

; search for all `publish` script methods that can publish messages
(globln "publish*" scriptMethods)

; create and publish 5x `SendEmail` Request DTOs for processing by TechStacks configured MQ Server
(doseq (to emails) (publishMessage "SendEmail" { :To to :Subject "Top 5 HN Links" :Body body }))

Run and watch Lisp Scripts

The same Sharp Scripts functionality for #Script is also available to Lisp scripts where you can use the web and app dotnet tools to run and watch stand-alone Lisp scripts with the .l file extension, e.g:

$ web run lisp.l
$ web watch lisp.l

To clarify the behavioural differences between the Lisp REPL's above which uses the same Lisp interpreter to maintain state changes across each command, the watch Script is run with a new Lisp Interpreter which starts with a fresh copy of the Global symbols table so any state changes after each Ctrl+S save point is discarded.

Watch `lisp` scripts

This quick demo illustrates the same functionality in Sharp Scripts is also available in lisp scripts where it provides instant feedback whilst you develop in real-time:

YouTube: youtu.be/rIgEP8ssikY

Annotated Lisp watch script

;<!--
; db sqlite
; db.connection northwind.sqlite
; files s3
; files.config {AccessKey:$AWS_S3_ACCESS_KEY,SecretKey:$AWS_S3_SECRET_KEY,Region:us-east-1,Bucket:rockwind}
;-->

; delete remove.txt file
(sh (str (if isWin "del" "rm") " remove.txt"))

; View all `northwind.sqlite` RDBMS Tables
(textDump (dbTableNames) { :caption "Northwind" } )

; Display first `customer` row in Single Row View showing all Table Columns
(textDump (dbSelect "select * from customer limit 1"))

; Display all Customers in London
(def city "London")
(textDump (dbSelect "select Id, CompanyName, ContactName from customer where city = @city" { :city city } ))

; View all root files and folders in configured S3 Virtual File Provider
(joinln (map #(str (.Name %) "/") (allRootDirectories vfsContent)))
(joinln (map .Name (allRootFiles vfsContent)))

; Show first 10 *.png files in S3 VFS Provider
(def pattern (or (first ARGV) "*.png"))
(joinln (map .VirtualPath (take 10 (findFiles vfsContent pattern))))

Page Arguments

You can also use the same syntax for declaring any app.settings page arguments used in #Script and code Scripts:

<!--
db sqlite
db.connection northwind.sqlite
-->

But for compatibility with any Lisp syntax highlighters and code editors they can also be prefixed with a ; line comment as seen above.

Executing Lisp in .NET

Lisp like all #Script languages are executed within a ScriptContext that defines all functionality available to them, i.e:

var context = new ScriptContext {
    Args = { ... },              // Global Arguments available to all Scripts, Pages, Partials, etc
    ScriptMethods = { ... },     // Additional Methods
    ScriptBlocks = { ... },      // Additional Script Blocks 
    FilterTransformers = { .. }, // Additional Stream Transformers
    PageFormats = { ... },       // Additional Text Document Formats
    Plugins = { ... },           // Encapsulated Features e.g. Markdown, Protected or ServiceStack Features

    ScanTypes = { ... },         // Auto register Methods, Blocks and Code Page Types
    ScanAssemblies = { ... },    // Auto register all Methods, Blocks and Code Page Types in Assembly
}.Init();

To render lisp you'll first need to register the Lisp Language with the ScriptContext you're using:

var context = new ScriptContext {
    ScriptLanguages = { ScriptLisp.Language }
}.Init();

Then use RenderLisp (i.e. instead of RenderScript) to render Lisp code, e.g:

// render lisp
var output = context.RenderLisp("(dateFormat now \"HH:mm:ss\")"); 

// async
var output = await context.RenderLispAsync("(dateFormat now \"HH:mm:ss\")");

These APIs match the high-level APIs for rendering normal #Script:

var output = context.RenderScript("{‎{ now |> dateFormat('HH:mm:ss') }‎}"); 
var output = await context.RenderScriptAsync("{‎{ now |> dateFormat('HH:mm:ss') }‎}");

Finer grained control

The high-level APIs above wraps the finer-grained functionality below which works by rendering a SharpPage configured with the lisp language in a PageResult that all languages use:

var context = new ScriptContext {
    ScriptLanguages = { ScriptLisp.Language }
}.Init();
var dynamicPage = context.LispSharpPage("(dateFormat now \"HH:mm:ss\")");          // render lisp
//var dynamicPage = context.SharpScriptPage("{‎{ now |> dateFormat('HH:mm:ss') }‎}"); // render #Script
var output = new PageResult(dynamicPage).RenderScript();

//async
var output = await new PageResult(dynamicPage).RenderScriptAsync();

Evaluating Lisp Script Results

If you instead wanted to access return values instead of its rendered output, use the EvaluateLisp() APIs:

var result = context.EvaluateLisp("(return (+ 1 1))"); //= 2

The generic overloads below utilizes ServiceStack's Auto Mapping utils to convert the return value into your preferred type, e.g:

double result = context.EvaluateLisp<double>("(return (+ 1 1))"); //= 2.0
string result = context.EvaluateLisp<string>("(return (+ 1 1))"); //= "2"

Which can also be used for more powerful conversions like converting an Object Dictionary into your preferred POCO:

var result = context.EvaluateLisp<Customer>(
    "(return (dbSingle \"select * from customer where id=@id\" { :id id }))",
    new ObjectDictionary {
        ["id"] = 1
    });

.NET Interop

The syntax for .NET Interop is inspired directly from Clojure's syntax used for Java Interop. See Scripting .NET Type Resolution for how to configure Types and imported Namespaces you want your Lisp scripts to have access to.

Member Access

The '.' prefix if for accessing an instance members which can be used for retrieving a properties public properties, fields and invoking instance methods, e.g:

(.Property instance)
(.Field instance)
(.Method instance ...args)

Indexer Access

Use ':' prefix for accessing a Types indexer or for indexing collections, e.g:

(:key indexer)
(:"string key" dictionary)
(:n list)
(:n array)
(:n enumerable)
(:n indexer)

It can also be used to access an instance public Properties and Fields:

(:Property instance)
(:Field instance)

However for readability we recommend using '.' prefix above to convey instance member access.

Constructor Access

Use '.' suffix for creating instances of Types:

(Type. ...args)
(Namespace.Type. ...args)

You can also create instances using the new script method, which as it accepts a string Type Name can be used to create generic classes with multiple generic args, e.g:

(new "Type" ...args)
(new "Type<T1,T2,T3>" ...args)

Static Member Access

Use the '/' separator to access a Type's static members or to invoke its static methods, e.g:

(StaticType/Property)
(StaticType/Field)
(StaticType/Const)
(StaticType/Method ...args)

Use '.' dot notation for specifying the fully-qualified Type name or to reference its Inner classes, e.g:

(Namespace.StaticType/Member)
(Namespace.StaticType.InnerType/Member)

Script Methods

Use '/' prefix to reference any Script Method registered in your ScriptContext:

(/scriptMethod ...args)

Script Methods without arguments can be referenced as an argument binding that when referenced as an argument (i.e. without brackets) are implicitly evaluated, in-effect making them a calculated property:

/methodAsBinding

While a '/' prefix indicates a reference to a script method, for readability it can be excluded as when there's no existing symbol defined in the Lisp interpreter's symbol table it will fallback to referencing a script method:

(scriptMethod ...args)
methodAsBinding

This does mean that when there exists a symbol of the same name defined you will need to use the '/' prefix to reference a script method.

Generic Types

All references above support referencing generic types and methods with a single generic Type argument, e.g:

(StaticType/Method<T>)
(GenericStaticType<T>/Member)
(GenericStaticType<T>/Method<T>)
(GenericType<T>.)

As ',' is one of Lisp's few syntax tokens (unquoting) it prevents them from being able to use them to specify multiple generic arguments.

Instead you'll need to use the Constructor function for referencing constructors with multiple generic arguments where you'll also need to specify the types of the exact constructor you want to call, e.g:

(/C "Tuple<String,int>(String,int)")

The difference between the /C script method constructor function and Lisp's C function is that the script method only returns a reference to the constructor which you'll need to invoke with arguments to create an instance:

((/C "Tuple<String,int>(String,int)") "A" 1)

Whilst Lisp's C function will auto-invoke the constructor function with the supplied arguments in a single expression:

(C "Tuple<String,int>(String,int)" "A" 1)

Likewise when needing to invoke generic methods with multiple generic args you'll need to use Function:

((/F "Tuple.Create<String,int>(String,int)") "A" 1)

Or Script Lisp's F function for invoking a function reference in a single expression:

(F "Tuple.Create<String,int>(String,int)" "A" 1)

For more examples and information see Scripting .NET Types.

Property Setters

You can populate multiple properties on an instance using the set script method, e.g:

(set instance )

Alternatively properties can be set individually with:

(.Prop instance arg)

Lisp Lists vs .NET Collections

A potential source of friction when interoperating with .NET is that Lisp Lists are Cons Cells so that a code or data list in Lisp, i.e:

'(1 2 3)
[1 2 3]

Is implemented as a Linked List of Cons cells:

(1 . (2 . (3 . null)))

Which is what Lisp's core functions expect to operate on, namely:

car cdr caar cadr cdar cddr caaar caadr cadar caddr cdaar cdadr cddar cdddr append mapcar consp cons?
listp list? memq member assq assoc nreverse last nconc dolist dotimes mapcan mapc nthcdr nbutlast

These core Lisp functions can't be used against .NET collections directly, instead you can use (to-cons collection) to convert a .NET IEnumerable collection into a cons list, e.g:

(cdr (to-cons netEnumerable))

Should you need to do the inverse you can use (to-list cons-list) to convert a cons list to a .NET List, e.g:

(to-list (range 10))

We've made Script Lisp's cons Cell an IEnumerable so that all other built-in Lisp functions can operate on both cons cells and .NET Collections where instead of iterating a list with (do-list) you can use (do-seq) to iterate both .NET Collections and cons cells, e.g:

(do-seq (x collection) (println x) )

Annotated .NET Interop Example

To see what this looks like in action here's an annotated simple real-world example that heavily utilizes .NET interop:

; define function and assign to `parse-rss` value in Lisp interpreters symbols table
(defn parse-rss [xml]
    ; define local variables used within this scope
    (let ( (to) (doc) (channel) (items) (el) )
        ; use XDocument.Parse() to parse xml string argument containing xml and assign to `doc`
        (def doc (System.Xml.Linq.XDocument/Parse xml))
        ; create empty ObjectDictionary (wrapper for Dictionary<string,object>) and assign to `to`
        (def to (ObjectDictionary.))
        ; create empty List of ObjectDictionary and assign to `items`
        (def items (List<ObjectDictionary>.))
        ; descend into first <channel> XML element and assign to `channel`
        (def channel (first (.Descendants doc "channel")))
        ; use `XLinqExtensions.FirstElement()` extension method to assign channels first XML element to `el`
        (def el (XLinqExtensions/FirstElement channel))

        ; iterate through all elements up to the first <item> and add them as top-level entries in `to`
        (while (not= (.LocalName (.Name el)) "item")
            ; add current XML element name and value entry to `to`
            (.Add to (.LocalName (.Name el)) (.Value el))
            ; move to next element using `XLinqExtensions.NextElement()` extension method
            (def el (XLinqExtensions/NextElement el)))

        ; add all rss <item>'s to `items` list
        ; iterate through all `channel` child <item> XML elements
        (doseq (elItem (.Descendants channel "item"))
            ; create empty ObjectDictionary and assign to `item`
            (def item (ObjectDictionary.))
            
            ; use `XLinqExtensions.FirstElement()` to assign <item> first XML element to `el`
            (def el (XLinqExtensions/FirstElement elItem))
            (while el
                ; add current XML element name and value entry to `item`
                (.Add item (.LocalName (.Name el)) (.Value el))
                ; move to next element using `XLinqExtensions.NextElement()` extension method
                (def el (XLinqExtensions/NextElement el)))

            ; add `item` ObjectDictionary to `items` List
            (.Add items item))

        ; add `items` ObjectDictionary List to `to` at key `items`
        (.Add to "items" items)
        ; return `to` ObjectDictionary
        to
    )
)

For comparison, this would be the equivalent implementation in C#:

public static ObjectDictionary ParseRss(string xml)
{
    var to = new ObjectDictionary();
    var items = new List<ObjectDictionary>();

    var doc = XDocument.Parse(xml);
    var channel = doc.Descendants("channel").First();
    var el = channel.FirstElement();
    while (el.Name != "item")
    {
        to[el.Name.LocalName] = el.Value;
        el = el.NextElement();
    }

    var elItems = channel.Descendants("item");
    foreach (var elItem in elItems)
    {
        var item = new ObjectDictionary();
        el = elItem.FirstElement();
        while (el != null)
        {
            item[el.Name.LocalName] = el.Value;
            el = el.NextElement();
        }

        items.Add(item);
    }

    to["items"] = items;
    return to;
}

Importing Global Scripts

Importing scripts in Lisp is essentially a 2-stage process of parsing Lisp source code into an SExpression, (basically Lisp's AST of tokenized elements captured in a 2-field Cons Cell) then evaluating it in a Lisp interpreter where any defined symbols are captured in its Symbols table.

Lisp Script captures its "standard library" in a Global Interpreter which serves as the starting template for all other Lisp Interpreters which starts with a copy of the Global symbols table which you can further populate with your own common functions using Lisp.Import(), e.g:

Lisp.Import(@"
(defun fib (n)
    (if (< n 2)
        1
        (+ (fib (- n 1))
        (fib (- n 2)) )))");

Loading Scripts

Loading scripts within a Lisp script works similarly except they're only loaded into that Lisp interpreters symbol table, a new one of which is created in each new PageResult.

Scripts loaded locally are loaded from the ScriptContext configured Virtual Files Provider which for #Script Pages SharpPagesFeature is configured to use the App's cascading virtual file sources.

A new ScriptContext starts with an empty MemoryVirtualFiles which you can write files to with:

var context = new ScriptContext {
    ScriptLanguages = { ScriptLisp.Language },
    ScriptMethods = { new ProtectedScripts() },
};
context.VirtualFiles.WriteFile("lib1.l", "(defn lib-calc [a b] (+ a b))");
context.VirtualFiles.WriteFile("/dir/lib2.l", "(defn lib-calc [a b] (* a b))");
context.Init();

You can load these scripts by symbol name where it assumes a .l extension, by quoting the argument so Lisp doesn't try to evaluate it as an argument, e.g:

(load 'lib1)

(lib-calc 4 5) ;= 9

Alternatively you can specify the virtual path to the script. You can load multiple scripts with the same definitions, in Lisp this just updates the value assigned to the symbol name with the latest definition, e.g:

(load "lib1.l")

(lib-calc 4 5) ;= 9

(load "/dir/lib2.l")

(lib-calc 4 5) ;= 20

Import Scripts from URLs

Inspired by Deno you can also import remote scripts from URLs, e.g:

(load "https://example.org/lib.l")

Locally Cached

Like Deno all remote resources are cached after first use so after it's loaded once it only loads the locally cached version of the script (where it will still work in an airplane without an internet connection). This cache is maintained under a .lisp folder at your configured Virtual Files provider (that can be deleted to clear any caches).

For Sharp Scripts or Apps using the web or app dotnet tools it's stored in its own cache folder that can be cleared with:

$ web --clean

Import Scripts from Gists

There's also first-class support for gists which you can reference with gist:<gist-id>, e.g:

(load "gist:2f14d629ba1852ee55865607f1fa2c3e")

This will load all gist files in gist order, if you only to load a single file from a gist you can specify it with:

(load "gist:2f14d629ba1852ee55865607f1fa2c3e/lib1.l")

Script Lisp Library Index

To provide human readable names to remote Lisp Scripts and a discoverable catalog where anyone can share their own scripts, you reference gists by name listed in the #Script Lisp Library Index which is itself a self-documenting machine and human readable gist of named links to external gists maintained by their respective authors.

Index library references can be loaded using the format index:<name>, e.g:

(load "index:lib-calc")

Which also support being able to reference individual gist files:

(load "index:lib-calc/lib1.l")

If you'd like to share your own Lisp Scripts with everyone and publish your library to the index, just add a link to your gist with your preferred name in the Gist Index Comments.

Viewing Script Source Code

You can view the source code of any load script references with load-src, e.g:

(load-src 'lib)
(load-src "/dir/lib2.l")
(load-src "https://example.org/lib.l")
(load-src "gist:2f14d629ba1852ee55865607f1fa2c3e/lib1.l")
(load-src "index:lib-calc")

Disable Remote Imports

Should you wish, you can prevent anyone from loading remote scripts with:

Lisp.AllowLoadingRemoteScripts = false;

#Script Pages Integration

Whilst Lisp is a small, powerfully expressive functional dynamic language it's not great for use as a templating language. Whilst there have been several attempts to create a HTML DSL in Lisp, nothing is better than having no syntax which is the default Template mode for #Script where it will emit everything that's not in a Template or Language Expression as verbatim text.

A nice USP of Script Lisp is that you're never forced into going "full Lisp", you can utilize #Script template expressions and Script Blocks handlebars syntax that provides the ideal DSL for usage in a Template Language for generating HTML and utilize your preferred Lisp or Code Script Languages for any computational logic you want included in your page using Language Blocks and Expressions.

Implementation

Despite being implemented in different languages a #Script page containing multiple languages, e.g:

Still only produces a single page AST, where when first loaded #Script parses the page contents as a contiguous ReadOnlyMemory<char> where page slices of any Language Blocks and Expressions on the page are delegated to the ScriptContext registered ScriptLanguages for parsing which returns a fragment which is added to the pages AST:

When executing the page, each language is responsible for rendering its own fragments which all write directly to the pages OutputStream to generate the pages output.

The multi-languages support in #Script is designed to be extensible where everything about the language is encapsulated within its ScriptLanguage implementation so that if you omit its registration:

    var context = new ScriptContext {
    //    ScriptLanguages = { ScriptLisp.Language }
    }.Init();

Any language expressions and language blocks referencing it become inert and its source code emitted as plain-text.

Lisp Argument Scopes

One differentiator between Lisp and Code languages is that code utilizes the containing page current scope for all its argument references where as Lisp stores all its definitions within the Lisp interpreter symbol table attached to the PageResult, so whilst Lisp scripts can access arguments within the pages scope, in order for the outer page to access any Lisp symbols they need to be exported, e.g:

Exporting Lisp Functions

Lisp functions can also be exported for usage in the rest of the page by calling (to-delegate lispFn) to convert it into a .NET Delegate, e.g:

Although an easier way to define functions in Lisp is to use the defn Script Block which wraps this in a convenient syntax:

Controlling Lisp output

One of Lisp's famous traits is everything is an expression which is typically desired within a language, but may not be what you want when generating output. E.g traditionally Lisp uses setq to set a variable which also returns its value that #Script will emit as it automatically emits all statement return values.

You could use def which is an alias for setq which returns null, other options include wrapping all statements within an empty let expression where only the last expression is returned, or you could use a Language Block Modifier to ignore the entire lisp block output and only export the result you want to be able to control precisely where to emit it:

can use either 'q', 'quiet' or 'mute' block modifier to ignore output

Another way to generate output from Lisp is to use its built-in print functions below:

(print ...args) - write all arguments to the OutputStream
(println ...args) - write all arguments to the OutputStream followed by a new line
(printlns ...args) - write all arguments to the OutputStream with a ' ' space delimiter followed by a new line
(pr ...args) - same as print but HTML encode all arguments
(prn ...args) - same as println but HTML encode all arguments

Learn #Script Lisp

A great resource for learning Script Lisp is seeing it in action by seeing how to implement C#'s 101 LINQ Examples in Lisp:

Lisp LINQ Examples

Explore APIs in real-time

We can also take advantage of Lisp's dynamism and interactivity to explore APIs in real-time, a great way to do this is via a watched Lisp script on the side where it provides instant feedback after each Ctrl+S save point or a active Lisp REPL.

symbols - List all symbols in Lisp interpreter - most symbols are named after standard Lisp or clojure functions
(symbol-type symbol) - Display the Symbol's Value Type
scriptMethods - List all available Script Method Names registered in ScriptContext
scriptMethodTypes - List all available Script Method Type information
(joinln collection) - Display the string output of each item in the collection on a separate line
(globln pattern collection) - Only display lines matching the glob pattern
(typeName instance) - View the instance Type Name
(props instance) - Display the Property names of an Instance public properties
(fields instance) - Display the Field names of an Instance public fields
(methods instance) - Display the Method names of an Instance public methods
(propTypes instance) - Get the PropertyInfo[] of an Instance public properties
(fieldTypes instance) - Get the FieldInfo[] of an Instance public fields
(methodTypes instance) - Get the Script Method Infos of an Instance public methods
(staticProps instance) - Display the Property names of an Instance public static properties
(staticFields instance) - Display the Field names of an Instance public static fields
(staticMethods instance) - Display the Method names of an Instance public static methods
(staticPropTypes instance) - Get the PropertyInfo[] of an Instance public static properties
(staticFieldTypes instance) - Get the FieldInfo[] of an Instance public static fields
(staticMethodTypes instance) - Get the Script Method Infos of an Instance public static methods

You can view the Scripts API Reference and Scripts Documentation on this website to interactively explore the available APIs, we'll work on providing further interactive documentation for the built-in Lisp functions, in the mean-time the best resource are their implementation.

For reference, here's are a quick list of all built-in Lisp symbols:

- % * *gensym-counter* *version* / /= _append _nreverse + < <= = > >= 1- 1+ 1st 2nd 3rd abs all? and any? 
append apply assoc assoc-key assoc-value assq atom atom? average butlast C caaar caadr caar cadar caddr 
cadr car cdaar cdadr cdar cddar cdddr cddr cdr ceiling cons cons? consp cos count debug dec decf def 
defmacro defn defun dispose dolist doseq doseq-while dotimes dump dump-inline elt empty? end? endp 
enumerator enumeratorCurrent enumeratorNext eq eql equal error even? every every? exit exp expt F f++ 
false filter filter-index first flatmap flatten floor gensym glob globln group-by htmldump identity if 
inc incf instance? intern isqrt it last length let letrec list list? listp load load-src logand logior 
logxor lower-case make-symbol map mapc mapcan mapcar map-index map-where max member memq min mod nbutlast 
nconc new new-map next not not= nreverse nth nthcdr null number? odd? or order-by pop pr prin1 princ 
print println printlns prn prs push push-end random range reduce remove remove-if rest return reverse 
round rplaca rplacd second seq? setcar setcdr set-difference sets sin skip skip-while some some? sort 
sort-by sqrt str string string? string-downcase string-upcase subseq sum symbol-name symbol-type t take 
take-while tan terpri textdump third to-array to-cons to-delegate to-dictionary to-list true truncate 
union unless upper-case when where where-index while zero? zerop zip zip-where

Common Lisp by convention uses a *p suffix for predicate functions but we prefer Clojure's (and Ruby's) more readable *? suffix convention, for source-code compatibility we include both for core Lisp predicts and just *? for others.

Happy Hacking!

We hope you enjoy the enhancments to #Script in this release and can't wait to see what you do with it. Any Questions or Feedback about this release is welcome on the Customer Forums.

Edit this page on GitHub

.NET Core 3
- Sync writes disabled by default
- Internal Changes
- Server Events Async APIs
- Upgrading to .NET Core 3.0
- Upgrading Docker and Travis CI to .NET Core 3.0
- Troubleshooting
- Upgrade to Fluent Validation v8.5
- Service Plugin APIs
- AuthFeature
OrmLite
- GetSchemaTable
- ServiceStack.Text
- Auto Mapping
#Script
- 1st class #Script Code and Lisp Language support!
- Transition to JS Pipeline Operator Syntax
- Scripting .NET!
Scripting .NET APIs
- Type Resolution
- Creating Instances
- Calling Methods
- Function
#Script Code
- Executing #Script Code in .NET
- Finer grained control
- Code Scripts
- Watched code scripts
Introducing #Script Lisp!
- Script Lisp Overview
- Lisp REPL
- Annotated REPL Walk through
- Lisp REPL TCP Server
- Password Protection
- Run and watch Lisp Scripts
- Watch
- Annotated Lisp watch script
Executing Lisp in .NET
- Finer grained control
- Evaluating Lisp Script Results
.NET Interop
- Member Access
- Indexer Access
- Constructor Access
- Static Member Access
- Script Methods
- Generic Types
- Property Setters
- Lisp Lists vs .NET Collections
- Annotated .NET Interop Example
- Importing Global Scripts
- Loading Scripts
- Import Scripts from URLs
- Locally Cached
- Import Scripts from Gists
- Script Lisp Library Index
- Viewing Script Source Code
- Disable Remote Imports
- #Script Pages Integration
- Lisp Argument Scopes
- Exporting Lisp Functions
- Controlling Lisp output
Learn #Script Lisp
- Explore APIs in real-time
Happy Hacking!

ServiceStack v5.7

.NET Core 3​

Sync writes disabled by default​

Internal Changes​

Server Events Async APIs​

Upgrading to .NET Core 3.0​

Upgrading Docker and Travis CI to .NET Core 3.0​

.travis.yml​

Dockerfile​

Troubleshooting​

Upgrade to Fluent Validation v8.5​

Service Plugin APIs​

AuthFeature​

/authenticate alias routes removed by default​

OrmLite​

GetSchemaTable​

DB Script Methods​

ServiceStack.Text​

Auto Mapping​

#Script​

1st class #Script Code and Lisp Language support!​

Transition to JS Pipeline Operator Syntax​

Scripting .NET!​

Script Assemblies and Types​

AllowScriptingOfAllTypes​

Scripting .NET APIs​

Type Resolution​

Examples Configuration​

Creating Instances​

Constructor Resolution​

Constructor function​

C() alias​

createInstance​

set​

Calling Methods​

call​

Method Resolution​

Generic Methods​

Function​

Instance Methods​

Static Type Methods​

Generic Static Type Methods​

F() alias​

Instance and Static Properties, Fields and Constants​

#Script Code​

Executing #Script Code in .NET​

Finer grained control​

Code Scripts​

Watched code scripts​

Introducing #Script Lisp!​

Script Lisp Overview​

Lisp REPL​

Annotated REPL Walk through​

Enable features and access resources with app.settings​

Lisp REPL TCP Server​

TechStacks TCP Lisp REPL Demo​

Password Protection​

Annotated Lisp TCP REPL Transcript​

Run and watch Lisp Scripts​

Watch lisp scripts​

Annotated Lisp watch script​

Page Arguments​

Executing Lisp in .NET​

Finer grained control​

Evaluating Lisp Script Results​

.NET Interop​

Member Access​

Indexer Access​

Constructor Access​

Static Member Access​

Script Methods​

Generic Types​

Property Setters​

Lisp Lists vs .NET Collections​

Annotated .NET Interop Example​

Importing Global Scripts​

Loading Scripts​

Import Scripts from URLs​

Locally Cached​

Import Scripts from Gists​

.NET Core 3

Sync writes disabled by default

Internal Changes

Server Events Async APIs

Upgrading to .NET Core 3.0

Upgrading Docker and Travis CI to .NET Core 3.0

.travis.yml

Dockerfile

Troubleshooting

Upgrade to Fluent Validation v8.5

Service Plugin APIs

AuthFeature

`/authenticate` alias routes removed by default

OrmLite

GetSchemaTable

DB Script Methods

ServiceStack.Text

Auto Mapping

#Script

1st class #Script Code and Lisp Language support!

Transition to JS Pipeline Operator Syntax

Scripting .NET!

Script Assemblies and Types

AllowScriptingOfAllTypes

Scripting .NET APIs

Type Resolution

Examples Configuration

Creating Instances

Constructor Resolution

Constructor function

C() alias

createInstance

set

Calling Methods

call

Method Resolution

Generic Methods

Function

Instance Methods

Static Type Methods

Generic Static Type Methods

F() alias

Instance and Static Properties, Fields and Constants

#Script Code

Executing #Script Code in .NET

Finer grained control

Code Scripts

Watched code scripts

Introducing #Script Lisp!

Script Lisp Overview

Lisp REPL

Annotated REPL Walk through

Enable features and access resources with app.settings

Lisp REPL TCP Server

TechStacks TCP Lisp REPL Demo

Password Protection

Annotated Lisp TCP REPL Transcript

Run and watch Lisp Scripts

Watch `lisp` scripts

Annotated Lisp watch script

Page Arguments

Executing Lisp in .NET

Finer grained control

Evaluating Lisp Script Results

.NET Interop

Member Access

Indexer Access

Constructor Access

Static Member Access

Script Methods

Generic Types

Property Setters

Lisp Lists vs .NET Collections

Annotated .NET Interop Example

Importing Global Scripts

Loading Scripts

Import Scripts from URLs

Locally Cached

Import Scripts from Gists

Script Lisp Library Index