Search Unity

Data save/load, RPC, transfer data to/from server

Discussion in 'Assets and Asset Store' started by McRain, May 9, 2011.

  1. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    After the May 25 "Unity Game Library" will not be available in AssetStore , because now ULIB client files will be available only in the full version ULIB.

    Forum thread about ULIB full version can be found here.



    Clients library which allows to serialize data from Unity3D to bytesArray and from a byteArray to objects Unity3D.

    Makes easy to save game data to a file or server , loading data runtime, the data transfer . Can be used to transfer any data between the game client through RPC.
    Allows the use of byteArray compression (for example, for mesh or large arrays) or encrypt

    Enclude:
    FileManager - class provides methods for loading, reading, searching, and saving files (include ini-file with section)
    LanguageManager - This class for add dynamic multilanguage support to game. Load language any time, change runtime.Provides methods for managing the list of translation, the translation of a key, etc.
    Serializer - provides methods for serializing and deserializing objects, compression, encryption.


    ULIB use to save data to file
    Not all objects can be serialized. But the list of available classes for serialization is very large. Full list of classes that can be serialized is written below. Even if the desired class of Unity is not listed - it does not mean that it can not be serialized, ULIB try to serialize it yourself. I am constantly working to increase this list. If what is your object can not be serialized - you can connect your own rules to handle the serialization and packing / unpacking your property yourself. To deserialize a class should be available in the assembly.
    Saving a file
    You just give away your property in ULIB and get ready to save the byte array. Then you can just store an array of bytes in the file. Or you can pre-compress it. Or you can pre-encrypt a byte array. Or you can encrypt and compress, or compress and encrypt.
    Reading from a file
    After reading from a file you get an array of bytes. This array of bytes you are sending in ULIB. Get back to your object. You do not need a special decryption or decompression. ULIB will determine the necessary operations and will give you a finished object. If your file is encrypted - you will need to specify keys.
    Recovery of references to objects
    When deserializing ULIB will try to recover the possible connection of objects that were to be serialized. This means that if your object in a variable was specified the same material - after deserialization of these variables will be listed on the same single instance of the material. This applies not only Unity classes.
    Send via RPC
    In RPC you can send a small set of types. ULIB can solve this problem. Now you can send the RPC, the types that are supported ULIB. To send data preparation for RPC use EncodeToString. To deserialize, use the feedback Decode (your string).
    Supported Types
    Simple Types boolean, integer, float, string
    Enumerators Array <T>, ArrayList, Hashtable, List <T>, Dictionary <K,V>
    Unity Classes Vector2/3/4, Quaternion, Rect, Color, Mesh, Texture2D, PhysicMaterial, GameObject, Material, MeshFilter, MeshRenderer, Transform, Collider, Light, Camera, Rigidbody, Projector, ParticleEmitter, and some subclasses of MonoBehaviour
    Other Subclasses of System.Object, Enum

    AssetStore : only $25

    Documentation (*.chm)

    Online documentation

    FileManager Description (en/ru)

    Support is provided by email support@reneos.com
     
    Last edited: May 14, 2012
  2. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    A small announcement features new versions of libraries for Unity3D games - ULIB:

    - Small time to serialize, about 5-10 times;

    - Added support for many Unity3D classes: Camera, Light (all types), Projector, Material, Texture2D, PhysicMaterial, Transform, Rigidbody, Collider (including MeshCollider, CapsuleCollider, SphereCollider, BoxCollider, WheelCollider), ParticleEmitter (base class);

    - Added support for serialization your own components (MonoBehaviour)

    - Added support for serialization Enum

    - A single instance of object: when the deserialization is now present as many copies of objects as they were before the serialization (for example, if two or more MeshRenderer.sharedMaterials contain the same material - after deserialize MeshRenderer.sharedMaterials will contain a one copy of the material ). This allows you to save a large number of copies of objects in a scene with little increase in file size;

    - Recovery of references to objects from scripts : if the script contains field and set to GameObject link or other object - after deserialization, this link will be restored;

    - Added the ability to encrypt stored files;

    In the near future a new version will be available on ULIB AssetStore.
     
  3. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    Library updated on Asset Store
     
  4. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    Added classes for working with files and translations.
    Added documentation.
    Added examples.

    See the new description in first post
     
  5. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
  6. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    New version send to AssetStore

    Changed examples - add dictionary save/load.
    Update LanguageManager and FileManager.
    Add support serilize GUITexture.
    Add ResourceManager.
    Remove small bugs.

     
  7. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    The new version LangaugeManager adds the ability to localize not only text but sound and texture. All the resources you can load from local files (png, jpg - graphics formats or ulib-format with compression and encryption). It is also available to download resources from the server. Through the use of a ResourceManager, the download occurs only once.
    It's very simple. You specify the source files (local or remote), a download of the available languages​​, the result as a OnGUI. Other everything for you do ULIB.
    Available translation by key, translation with the replacement tag, texture for the current language and other features LanguageManager, such as partial loading of the translation for the current module / scenes, etc.

     
  8. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    The library ULIB (client) updated on AssetStore to version 4.4.0.1



    The new version adds new manager for fast development of games, fixed bugs , changes in the serialization algorithm, adds the ability to use external serializers.

    Price change to $30.

    Detailed documentation is being prepared and will be available within a few days.
    Thank you.
     
  9. demonicmetal cs

    demonicmetal cs

    Joined:
    Feb 11, 2010
    Posts:
    68
    when I open de file en try to fill my List<Item> i get this error (221)DecodeObject can't create Instance of CraftItem

    CraftItem inherits form Item

    Code (csharp):
    1. using UnityEngine;
    2. using System.Collections;
    3.  
    4. public class CraftItem : Item {
    5.  
    6.     public CraftItem(int id,string name,string[] descriptions ,Texture2D texture,GameObject prefab){
    7.         base.id = id;
    8.         base.name =  name;
    9.         base.descriptions = descriptions;
    10.         base.texture = texture;
    11.         base.prefab = prefab;
    12.     }
    13.    
    14.    
    15. }
    Code (csharp):
    1. using UnityEngine;
    2. using System.Collections;
    3. using System.Collections.Generic;
    4.  
    5. public class Item  {
    6.  
    7.     public int id ;
    8.     public string name = "";
    9.     public string[] descriptions = new string[3]{"","",""};
    10.     public Texture2D texture;
    11.     public GameObject prefab;
    12.    
    13.     public Item(){
    14.     }
    15.     public Item(int id,string name,string[] descriptions ,Texture2D texture,GameObject prefab){
    16.         this.id = id;
    17.         this.name =  name;
    18.         this.descriptions = descriptions;
    19.         this.texture = texture;
    20.         this.prefab = prefab;
    21.     }
    22. }
    23.  
    Any idea how to fix this ??

    ps.
    its a great scripting tool
     
  10. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    Hi!
    I think that the problem lies in the class constructor. In this version of ULIB you can use only constructor with no parameters.
    Thank you.
     
  11. demonicmetal cs

    demonicmetal cs

    Joined:
    Feb 11, 2010
    Posts:
    68
    thank that solves my problem
     
  12. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    Hi.
    For those who want to save (or send) other data (as opposed to ULIB) or who want to extended ULIB in your project - I hope this method should work.
    As well as processing components, processing of objects can also be expanded or modified.
    You can replace the function Mesh what used in ULIB on your own function.

    1. Create a function for encode Mesh by example:

    Code (csharp):
    1. byte[] MyOwnMeshEncoder(object val)
    2.     {
    3.         var ms = (Mesh) val;
    4.        //you can used list of property what you want
    5.         var table = new Hashtable
    6.                                                {
    7.                                                    {"vertices",ms.vertices},
    8.                                                    {"uv",ms.uv},
    9.                                                    {"triangles",ms.triangles},
    10.                                                    {"tangents",ms.tangents},
    11.                                                    {"normals",ms.normals}
    12.                                                };
    13.         var dataBytes = _serializer.EncodeHashtable(table);
    14.         var dataBytesLength = dataBytes.Length;
    15.         var result = new byte[1 + dataBytesLength];
    16.         result[0] = 22;//this is code for Mesh!!
    17.         Buffer.BlockCopy(dataBytes, 0, result, 1, dataBytesLength);
    18.         return result;
    19.     }
    2. Create a function for decode Mesh by example.

    Code (csharp):
    1. object MyOwnMeshDecoder(byte[] inBytes, ref int startPos)
    2.     {
    3.         var mesh = new Mesh();
    4.         var table = _serializer.DecodeHashtable(inBytes, ref startPos);
    5.         mesh.vertices = (Vector3[]) table["vertices"];
    6.         mesh.uv = (Vector2[]) table["uv"];
    7.         mesh.triangles = (int[])table["triangles"];
    8.         mesh.tangents = (Vector4[]) table["tangents"];
    9.         mesh.normals = (Vector3[]) table["normals"];
    10.         return mesh;
    11.     }

    3. Add these to the code before (ULIB first seeking external functions ).

    Code (csharp):
    1. Serializer.TypeEncoders.Add(typeof(Mesh), MyOwnMeshEncoder);
    2. Serializer.TypeDecoders.Add(22, MyOwnMeshDecoder);

    Or you can override this example for used your own list of properties for extend ULIB.

    Code for used:

    22 - Mesh
    23 - Texture2D
    24 - Physics
    25 - GameObject
    26 - Material


    You can use your code for your class - please used from 150 to 199. (this opportunity only for test now, sorry)

    Please - not used this way if you send data to / from ULIB-PHP ! Now this only for local.

    If there are difficulties or questions - please let me know.

    Thank you.

    P.S. No need to update ULIB - this method should work in the current latest version.
     
  13. TouchSoft

    TouchSoft

    Joined:
    Oct 18, 2008
    Posts:
    218
    Does this work for mobile platforms?
     
  14. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    Yes, it works on mobile platforms. I tested the work on Android, and buyers have written to me that it works on iOS.
     
  15. TouchSoft

    TouchSoft

    Joined:
    Oct 18, 2008
    Posts:
    218
    Awesome thanks! I'll be getting this!
     
  16. kenlem

    kenlem

    Joined:
    Oct 16, 2008
    Posts:
    1,630
    Just purchased and really enjoyed playing with it. Nice work and I'm glad I bought it.
     
    Last edited: Feb 28, 2012
  17. kenlem

    kenlem

    Joined:
    Oct 16, 2008
    Posts:
    1,630
    Doesn't seem to serialize the velocity of a game object. Is that possible?
     
  18. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    Yes, it can be.
    ULIB does not support all components Unity3d.
    But you can always expand (or change) the rules of serialization - please see the examples from "example_used_external.unity".
    Also, you can read help from my blog: http://game.reneos.com/blog/archives/1207
    In any case - if you can not solve it by yourself - please send me an email with a description of the problem: support@reneos.com
    Thank you.
     
  19. kenlem

    kenlem

    Joined:
    Oct 16, 2008
    Posts:
    1,630
    Thanks for the reply. I'll have a go at it.

    I do see that ULib supports rigidbodies but that probably only supports the public properties of a rigidbody.

    So the correct TypeDecorder would be 24 Phyics? Makes me wish I bought the source code version.
     
  20. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    ULIB read fields and property from object only if ULIB can set them later (not readOnly)
    Please send me a part of code that causes problem, I need to understand how I can help you.
    Thank you.
     
  21. kenlem

    kenlem

    Joined:
    Oct 16, 2008
    Posts:
    1,630
    I sent you a project with an example.

    I did just notice that my example is only encoding and decoding the GameObject. I am assuming that any components on the GameObject will automatically be serialized as well. Should I be explicitly serializing the Rigidbody of the GameObject as well?
     
  22. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    Yeah, I got your example and I now understand your problem.
    ULIB serializes Rigidbody ULIB but not serialize PhysicsMaterial.
    You need to add serialization PhysicsMaterial - in Serializer.TypeEncoders.
    I will prepare and send to you my example tomorrow afternoon.
    Thank you.
     
  23. kenlem

    kenlem

    Joined:
    Oct 16, 2008
    Posts:
    1,630
    Hi McRain,

    I really don't think it has to do with the physics material at all. I took out the Phyics material in my example and it didn't change anything.

    The behavior I am seeing is that if I save a gameObject with a rigidbody and that rigid body has some velocity to it when I save it, those velocity values are not being set when I create the game object from memory.

    I understand that you don't serialize any properties that you can't set when you create the new object but you can set velocity and angular velocity of a rigid body directly. Those are not read only fields.

    Does your $70 version include source?
     
  24. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    Yes, these fields are not used - may be these properties have been
    added later in Rigidbody.
    This will be fixed in the new version.
    $70 version includes a PHP server - no other differences.
    Thank you.
     
  25. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    In ULIB there is a possibility to extend serialization components. By – default, if components are not yet supported ULIB serialized as UnknowComponents – that is, information and data include all public fields and properties, including inherited members.

    If you need to reduce the list of data used – you can extend ULIB with your own method of serialization.

    Please read here: http://game.reneos.com/blog/archives/1272
     
  26. kenlem

    kenlem

    Joined:
    Oct 16, 2008
    Posts:
    1,630
    Hi,

    I just realized that the Serializer requires Texture2D objects to be or a certain format and to be readable/writeable because it actually serializes the texture data. While this is fine for certain cases where you are downloading resources from an external source or saving on a desktop, it would end up using a lot of memory on a mobile device.

    Using your directions, I wrote an encoder/decoder that saves on the name of the Texture2D so that when the object is deserialized, it just finds the instance of the correct texture from the resources folder. The only slight drawback is that the texture needs to be located in the Resources folder. I don't really consider that much of a drawback.

    It seems to work.

    Code (csharp):
    1.  
    2.  
    3.         Serializer.TypeEncoders.Add(typeof(Texture2D), EncodeTexture2D);
    4.         Serializer.TypeDecoders.Add(23, DecodeTexture2D);
    5.  
    Code (csharp):
    1.  
    2.     byte[] EncodeTexture2D(object inObject)
    3.     {
    4.  
    5.         Debug.Log("Encode Texture2D");
    6.        
    7.         Texture2D texture2d = (Texture2D)inObject;
    8.  
    9.         var table = new Hashtable
    10.            {
    11.  
    12.                {"name",texture2d.name},
    13.  
    14.            };
    15.  
    16.         var dataBytes = _serializer.EncodeHashtable(table);
    17.  
    18.         var dataBytesLength = dataBytes.Length;
    19.  
    20.         var result = new byte[1 + dataBytesLength];
    21.  
    22.         result[0] = 0x17; //this is code for Texture2D
    23.  
    24.         Buffer.BlockCopy(dataBytes, 0, result, 1, dataBytesLength);
    25.  
    26.         return result;
    27.     }
    28.  
    29.  
    30.     object DecodeTexture2D(byte[] inBytes, ref int startPos)
    31.     {
    32.         Debug.Log("Decode Texture2D");
    33.        
    34.         var table = _serializer.DecodeHashtable(inBytes, ref startPos);
    35.  
    36.         String name = (String) table["name"];
    37.  
    38.        
    39.         return (object)Resources.Load(name);
    40.     }
    41.  
     
  27. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    Thank you.
    The new version ULIB provide an opportunity for just such an opportunity. In the new version you will be able to create the code in a dll as an external plug-in.
    This is a couple of days.)
     
  28. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    Very soon, will update the library: new functions
    - You can use ULIB sender of the gateway, or use your own.
    - Plugin system allows you to expand or change the serializer
    - You can easily use own plugin system for your game
    - And more

    See the video example

     
  29. kenlem

    kenlem

    Joined:
    Oct 16, 2008
    Posts:
    1,630
    Can't wait to check it out.

    Tried to watch your video but 22 minutes without audio is much for me! I'll check it out once I download it.
     
  30. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    I can voice the video, but my English is not nobody understand, sorry )))
    I hope that the example scripts enough to understand)
     
  31. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    Unity Game Library version 4.5.1b sent to AssetStore. What's new:
    -Ability to create your own plugin system for your game;
    -You can do serialization of components and objects with your own methods;
    -Faster serialization algorithm;
    -You can use methods of Serializer (EncodeHashtable, EncodeArray,EncodeArrayList etc);
    -Changed the serialization of components for compatibility with the latest version of Unity3D;
    -And other changes.

    Successful development!
     
  32. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
  33. ddt

    ddt

    Joined:
    Sep 14, 2010
    Posts:
    8
    I'm on iOS and trying to load PNG files from storage to memory and from memory to GPU, all asynchronously. I can do this with AssetBundles if I import them, but then the importer appears to decompress the PNG, recompress it, and swell it up in the process. Is there a way to skip that part?

    I'm on the prowl for a plugin that will do this asynchronously. Will FileManager do this?

    Thanks!

    advice@gmail.com
     
  34. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    Hi.

    You can save your texture as an array of Color [] (the editor). Later, at runtime, you can load array from file and create the texture.

    (Now I am unable to test this in iOS, unfortunately.)

    SAVE
    Code (csharp):
    1. var txtr = (Texture2D)sourceTransform.GetComponent<MeshRenderer>().sharedMaterial.mainTexture;
    2.         var colors = txtr.GetPixels();
    3.         FileManager.SaveObject(Application.dataPath +"/texturefile.unp",colors);
    LOAD
    Code (csharp):
    1. var colors = (Color[])FileManager.LoadObject(Application.dataPath +"/texturefile.unp");
    2.         var txtr = new Texture2D(64,64);
    3.         txtr.SetPixels(colors);
    4.         txtr.Apply();
    5.         sourceTransform.GetComponent<MeshRenderer>().sharedMaterial.mainTexture = txtr;
    Or you can save size of texture to file too.

    SAVE
    Code (csharp):
    1. var txtr = (Texture2D)sourceTransform.GetComponent<MeshRenderer>().sharedMaterial.mainTexture;
    2.         var colors = txtr.GetPixels();
    3.         FileManager.SaveObject(Application.dataPath +"/texturefile.unp", new Hashtable() { { "w", txtr.width }, { "h", txtr.height }, { "c", colors } });
    LOAD
    Code (csharp):
    1.  var table =
    2.             (Hashtable)
    3.             FileManager.LoadObject(Application.dataPath +"/texturefile.unp");
    4.         var colors = (Color[])table["c"];
    5.         var txtr = new Texture2D((int)table["w"],(int)table["h"]);
    6.         txtr.SetPixels(colors);
    7.         txtr.Apply();
    8.         sourceTransform.GetComponent<MeshRenderer>().sharedMaterial.mainTexture = txtr;
    Thank you.
     
  35. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    Price lowered for you : only 25$ :D
     
  36. McRain

    McRain

    Joined:
    Mar 26, 2009
    Posts:
    138
    After the May 20 "Unity Game Library" will not be available in AssetStore , because now ULIB client files will be available only in the full version ULIB.
    Forum thread about ULIB full version can be found here.