Asset Store Quality Control?

Hi all,

do you think that unity should have some quality control in the asset store?

the thing is that I just bought a package ($50) and there are so many bugs that the package is unusable.. the developer is not updating it anymore so I just contacted the unity support to ask for a refund but it seems unity don't give refund:

Hi There,

Thanks for the e-mail.

Thank you for contacting us. As specified in the Asset Store EULA, which all Asset Store customers have agreed to, all sales are final. However you may wish to contact the creator or publisher of the asset for their assistance. In some cases, they may be willing to authorize a refund at their discretion.

You must speak with the publisher, only they can authorize refunds by requesting the refund on your behalf.

Kind regards,
Ben
Unity Support

So, no developer support, no unity support, you are at your own..

I think that unity have to check the quality of the assets they approve to be sold in the asset store at least we deserve that.

 

Yes absolutely.

When buying assets I always do as much research on it as possible. Look at the date it was last updated. Look at the reviews, etc. Generally if it hasn't been updated in over a year, it's a deal breaker for me, unless it's a really simple basic thing

Maybe not so much a 'quality control' but more of a 'report this package is bugged and does not work in the current version of Unity' system?

 

Yea, I also bumped into a couple of packages that are pretty much unusable now because they are so outdated.

 

UT does go through an approval process before adding a package to the asset store. I don't know how deep they test complex code packages, but if there are any crashes or console errors, it will get rejected. It's probably a little more difficult to spot individual bugs, as that requires learning and using the product beyond the example scenes.
As for reporting that it doesn't work in the current version, that would be great except for the fact that many people still use those older versions

 

stick to well reputed packages, there are 20-25 worth a look and consideration ... all the others are unknown quantities that you buy at your risk

 

He asked for a refund because it was broken. If that doesn't count as 'reporting' it, what does? Why would a customer have to do more than that?

 

I just got an email from an artist 3 weeks after I wrote but this was for wanting to buy more art. In any case, in places outside the US the workers typically get from 4 - 6 weeks vacation not the 1 or 2 weeks in the US per year and if they have any sense don't take their email with them.

If they don't answer in 7 weeks I'd consider the asset abandoned so your should fix the asset and say you are going to release it under an MIT license which you can do. That will get a response if they are ignoring you in order to keep $33.33.

 

The one easiest things UT can do to help in that regard is to refuse assets using a language other than C# as importing multiple JS scripts from different assets is an easy way to clobber your project and require those that use C# to one or more namespaces that includes the Asset Kiosk owner's Kiosk / Real Name as part of the organizing and isolating the asset code from other assets and Unity code.

Deprecated API? Well if an Asset hasn't been updated in 2 cycles and has deprecated API calls it needs to come with a yellow flag, 'Deprecated', rather than 'Sale' for buyers or removed. I still want a filter / cookie I can set in my account information that will filter out all packages by an arbitrary date I set, by version number, by content (Do I really want to scroll through pages of zombie and gun art assets? No.), and to allowed me to hide purchased packages that I likely never use.

It's not like my IE 11 browser doesn't already struggle for 5 minutes waiting for scripts to finish when I go to the asset store.

 

Not legally, you can't.

 

Then what? You have to wait 75 years? I'd file a complaint then with PayPal or the Credit Card company and try to get charges reversed. Just because something is 'software' and has evasive EULAs written with the express intent of avoiding culpability for your business' work rather than protecting against an honest mistake should not give license to throw sh*t against the wall and see what sticks.

 

I think that the best way of helping people related to obsolete content in the Asset Store would be the following things :

- Adding a filter/mention which include what versions of Unity the asset have been tested with.
Whenever someone add an asset to the store, he will have to select which one is the latest version in which the asset have been tested.
So, for example, if someone produced an asset for Unity 4.1 and haven't updated it since then, on the asset store, it should be shown as "Tested on Unity 4.1". If an update for a new version is done, then that someone could update it to "Tested on Unity 4.3"

- Adding a Incompatible report button as well as a report graph on the asset store pages
My idea of this button would be a button that, upon pressing it, bring a small menu on which the user must enter what version of Unity he is using.
Then, once he entered his version of Unity and confirmed his report, this would update a small graphic which would give the total of people who have not been able to uses the asset with the recent versions. Since there are a load bunch of versions, it could be limited to 4 versions with a 5th one being "Other previous version". So, with the current Unity 4.3.4, this graph would show something like :
Number of incompatible reports :
Unity 4.3.4 = X
Unity 4.3.3 = X
Unity 4.3.2 = X
Unity 4.3.1 = X
Older versions = X

Note that only those who bought it should be able to report. (So that no hateful report should be done)
But, those who bought it can report it as incompatible for multiple version if they tested it.

This would allow :
- Customers to have an idea about if the asset is compatible with their current version of Unity.
- Whenever a new version come in, this would allow the system to quickly update the list by adding the 4th version on the "Older version" count.
- Since this cover 4 versions, this would also allow older projects (since project can run for 3-4 years) to have an idea if the asset is viable or not.
- If a project is having too many reports for the latest version or even worse, multiple lasted versions, this would allow UT's people to retest older content (at a respectable speed) and, if needed, send an automatic report to the asset owner to inform him/her that its asset should be updated or might be removed from the Asset Store.
- It's not that rare that group/company/individual uses 2 versions of Unity. One might be for an older project which might have some problem if updated while the other is a newer project which uses the new features. This option would also allow them to be warned that an asset will only be compatible with one of their project.

That's just my suggestion for the issue.

 

I have many, many assets that have deprecated calls and they aren't being updated although I know the asset store owner is still active. They need to update their code or we need a button to report the deprecated calls to the asset store review system. One review can be a self written view that can be 'argued' to be subjective although often they are objective and the other a purely technical review that is objective and the number of deprecated and broken lines of code is as objective a review as any. ...and add namespace conflicts to that list of objective measures of quality.

 

Totally agree

 

I'd love to see something like that, but many authors *do* update their assets to be compatible eventually. The system would need some way for them to erase the bad marks against their asset if they update it to work. But by the same token, that process shouldn't mean that Unity Tech has to do any testing, because that's just expensive for them.

Perhaps the reports could be against versions of the product, and could also include reports that it *does* work.

 

Indeed, anyone who owned the asset could be given an update button in their download center that indicated for that asset that if they had reported that it was using deprecated functions, they could check that and remove their report, assuming they had any sense of decency that should be a nice system, esp if it gets a liked asset working again

A more comprehensive set of filters (That can be combined) to the asset store would improve it immensely, there aren't enough words for how much it would make it easier to find, what you want, little known highly useful assets become easier to find (And profitability hence much better spread and fair to asset creators), and just generally nicer and improving productivity immensely. How hard can that even be?

 

No they wouldn't because that metric is also a measure of how long it takes them to update to compatibility. I have no interest in the update after I've bought a competitive product that updates promptly.

 

For what it's worth, I've not had an issue getting refunds in the past. Also, when someone once had an "issue" with something I'd made (nothing was broken, they had just changed their mind) and complained to Unity that I couldn't be contacted (I'd got one email less than 48 hours before over a weekend) Unity got in contact with me within 48 hours to see what was up.

So, while I know that the written policy is pretty black and white, my experience suggests that in day to day dealings Unity's staff are actually far more flexible and reasonable than the policy implies.

I do agree that higher quality standards would rock. But for the most part the system that's there is quite workable. My main concern personally is wanting to check out code assets before purchasing them. There's a fair bit of stuff out there that works but isn't well engineered, which means that as a drag-and-drop it's fine but it can be a pain integrating it into a larger work. That's the main area where I see Asset Store stuff being sketchy - code quality isn't really being promised, so it's a risk that you have to accept prior to purchase.

 

Unity really needs to make some guidelines for sellers.

I'm so sick of importing packages and having generic duplicate scripts even from the same author. Mouselook, etc. Unity needs to either enforce namespaces or require scripts to be prefixed with the product's initials or something.

Now that I've got the plugin "Grab yer assets" I can import 30-50 unitypackages at a time and I just get bombarded with 50+ errors for duplicate scripts and have to spend a lot of time going through them and renaming, every freakin time I import assets....Same with the image effects(pro) folders, if I import several assets, each one has the same folder with the same scripts.

 

This could be an easy fix if Unity required namespace registration for an asset store submission... AND not permitting anything (scripts, assets.. w/e) outside that namespace folder within the package. simple.

 

Unfortunately it is not simple. Most publishers, including myself are using Unity's standard assets to demonstrate the actual package. Many are using the first person setup to allow the customer to walk around. If one wants that it looks a little nicer, the image effects are also needed. What are we supposed to do?

 

You change the name of the scripts or add a namespace to them in the top of the script.......simply rename it in the editor, then dbl click to open in monodevelop and make sure the classname is the same as the filename. So if you make xyz asset make the name xyz_Mouselook.cs or something. Or add a namespace.

If asset sellers would do this it was save their customers a lot of time. You do it once, rather than every customer having to do it themselves...



I wonder if it's possible to write a plugin to parse all the scripts and add namespace to the classes....?

 

Even if I try to do my best to avoid conflicts, I would never ever rename all the image effect scripts and shaders and what not. Those are just demo files that are used to to show something in a nicer outfit. They don't belong in any actual project! Renaming them and or adding them to a namespace would just be a lot of wasted time and another time waste if Unity updates those assets and so on.
My idea behind the demos folder is that when someone tries the package out, they can check the demos, but if a customer uses it in an actual project, they should delete the demo folder. In previous versions I had the demos in a unitypackage, but after several rejections, I switched to using a folder that can be deleted. All the my code in the asset store is using namespaces with the exception of a handful of classes that has prefixes and is not in a namespace to preserve the compatibility.

I don't know if it is more difficult with other projects from the asset store to get rid of the demos, but I really believe that it shouldn't be too much for a customer to get rid of one folder or to not even import that folder into a project.

 

any standard assets used are already inclusive with unity, so repackaging them seems un-necessary... I understand your required to do so currently, because there is no method during submission to perhaps "check" a dependency for your assets' use. If you had the ability during submission to check "standard assets are required", then during the Unity install process it automagically includes those dependencies.. saving you the trouble of distributing them with the asset, and perhaps force standardization.

I'm just thinking logically on how it could be improved in theory, but people like yourself have real experience so Unity should be listening to you on how to improve it to avoid conflicts and duplication in distribution.

(btw, you and/or your team has made some wonderful assets)

 

Always a risk buying code assets. Same not really true for art assets (I think). I am not an artist Anyways, buyer basically has no idea how crappy or buggy it will be until they actually buy it. This is why it should be obvious to every buyer that if the asset is not updated in a long time / has crappy reviews then you should not expect quality.

 

Either rewrite the scripts to put them in a separate name space or better would be to leave them in the original location when you import them from the Unity standard / pro assets. The Unity user importing those standard / Pro assets directly and clobbering those imported by your package is desired over rewriting those scripts to use different namespace if you don't want problems with namespaces and extra support and maintenance later.

The asset buyer can't simply look though the asset package list to uncheck scripts that look like they may be copies of Unity scripts because the asset maker may have made significant changes to the script or it may even be an original script. It seems required Unity assets in the default location when originally imported by the asset maker and then packaged in that same location is the sensible way to avoid this.

Is there something in Unity's asset store rules that says Unity assets must be moved to reside under an original unique folder name that contains all the non-Unity parts of the asset plus the Unity parts of the asset? Because I'm often seeing several folders imported in many assets and those names are sometimes common, e.g. resources.

Anyway, that's what CM is for.

 

I personally don't see a way how it would be possible to find the dependencies under all circumstances. In my opinion the easiest way would be to split the demos from the actual assets. When you downloaded the asset, it would be handy to have the option to also download/import the demos. In an actual production project, only the assets would need to be imported which should not lead to any conflicts.

Thanks a lot!

 

I bought an asset yesterday that included 5 very good example projects that you had to download separately (meaning they had to host the archives too) and the actual asset was a very quick download. They also have excellent Unity web players of their assets.

 

The idea is that reviews are there to help you make an informed purchase. This sometimes helps, but has some problems:

1. Too many glowing 5 star reviews or negative 1 star reviews.
2. The people writing the reviews might not have your same requirements. They might have a different application size or a different level of production value from you.
3. Too few reviews for some products, so being any early-adopter means you are taking a risk.

 

I completely agree with this. Name conflicts are a huge pain. It's not just duplicate scripts, either, it's also use of common class names like "Localisation".

 

I don't think it shouldn't be required for an asset store publisher to host the examples themselves.

Edit: Not every publisher has the infrastructure for that, while the asset store already has it.

 

I saw somewhere (I think the NGUI forums) someone shared a snippet to do exactly that, but they said a little manual handling was still required. Though I'm sure it could be improved upon. It'd need to:
- Only modify the new stuff
- Do some kind of magic to rename files while retaining any references

 

I agree that the ratings are not very helpful. It very much feels as if the people who kind of like something give 5 stars, while the others who may not feel like it, give 1 star. The overall value in those ratings is questionable from my point of view. It neither serves potential customers, not publishers.
On the other hand, there is a lot more value if there is a review. Regarding the quality of ratings, it would help in my opinion to only allow ratings in combination with a review. This would heavily lower the amount of ratings, but would most likely increase their value.

 

In my experience, I've found the average rating of a product to give a fairly accurate, general indication of whether that product is at least usable or not.
When there aren't many reviews to go on, you've gotta take a close look at the demo videos/webplayers/anything else you can in order to make an informed decision.

 

Yes there is a definite case of quantity over quality which I think gives the asset store a bad image.

There are also some promising assets that get released in a 'beta' state and then get abandoned or never completed but remain on sale - e.g. Livity is almost there but has lost support and bug fixes.
Have reported

 

This is a basic problem with ratings and reviews though. People who really like it or really hate it are more motivated to post a review or rating. Someone who is "meh" about the product may not care enough to take the time to put down a three-star rating.

 

The asset store doesn't have that infrastructure, I've asked several times to create self contained 'Asset Kiosks' for each publisher with Kiosk emails, kiosk support forums, and all the minutia you'd expect from a professional business product instead of the completely amateur system in place now where if I need to ask for support I'm looking up the product in the asset store and maybe sending email to a google account, to another company domain account or visiting a wix or word press or company domain style web sites and the assets are sometimes under personal names and sometimes under business names and sometimes under most worrying of all, 'forum style handles' and sometimes all used interchangeable these things by the same publisher.

 

This is mostly only a problem for demonstrating art assets. Code assets are almost always put in their own namespace if the developer values time off.

 

It would be nice if it was easy to create a snap shot of an asset in action and post than and state created with asset x. I've written a couple of critiques of an asset in that asset's support thread but of course I didn't go back to the asset store expressly to write a review based on that asset's shortcoming I voiced would be nice to see remedied in future versions.

Note: my suggestions to the asset developer were not 'feature creep' but adding real simulation calculations to improve a product that was doing simulations and left out a key part of the simulation needed to create a more realistic simulation. Bizarrely, another asset purchaser told me I was free to develop all the features that asset was missing as improvements myself rather than those being sensibly be developed by the asset developer that is profiting from sales of the asset to myself and others.

Had the asset store self contained 'Kiosk' for each Asset Store seller then you could easily read the support quests of the asset in the owner's Kiosk tab. And by doing that they could have each post to the Kiosk be tagged as a review or a support request / question / or an answer and then use sorting and filtering to create the 'support' and 'review' sections of the Kiosk front.

They'd also be able to track the number of support requests made vs. the support requests answered / closed. Closed because sometimes requests aren't valid, sometimes they aren't supported (i.e. we stopped support Unity 3.5 months ago). That won't stop an asset store seller from closing support requests that are legitimate and haven't been resolved but then that's were legitimate bad reviews have value and not simply purchasing with false expectations.

And they need to do something to the script code slowing the browser to a crawl while some process is inspecting if it has Trojans and such. It is so sssssllllllllooooowwwww (within Unity itself, in IE11, in Chrome, using Windows Defender on Windows 8.1 64 bit).

 

With the infrastructure I meant as server from which assets can be downloaded. It wouldn't be that difficult for Unity to have a download asset and another download demos button. This would be a very small change for them, but it would help many customers and also the publishers.

 

Yes, the Asset Store does need quality control, but in other fields.

1) Depreciated assets, that simple don't work because they are too old, should be removed. Yes, there's a one-in-million chance, that some person out there needs that old version because he or she works with an age-old version of Unity. But I doubt that this old asset was so ingenious that it's functionality couldn't be replaced. Besides, a cleaner Asset Store might be worth disappointing five people a year.

2) Removal of just bad assets. Yes, we are now entering the mine field of "what is good artwork?". On the other hand you can argue that "you know it when you see it". There will always be a bit of benevolent dictatorship going on, so why not let a team of people decide, and refuse an asset once in a while? Bottom line is: the Unity Asset Store sponsors some amazing work, but I've also seen assets on the Asset Store that make the store look bad.

3) Duplicate scripts and lack of namespaces are a problem. One solution: Disallow Unity Standard assets in Unity Asset Store packages. No need to include freely available assets. Just make it very clear in the description what is required. Generate a visual dependence tree if necessary. This is also related to assets installing themselves all over the place instead of in folders a la Editor or Plugins. Dito generic folder names (Models/Ships/... instead of AssetProvider/AssetName/Models/Ships.../).

4) More compatibility criteria than just Pro/Free. Platforms matter. This was discussed recently.

5) In my opinion there should be a rating for a publisher. That might make it easier for a established publisher to introduce a new product, that he or she is not yet known for.

6) There should be ratings for reviewers. See Amazon ("trusted reviewer"). This is a little bit there right now by voting on reviews ("helpfulness") but it could go further.

7) Rating inflation is very very real in the Asset Store. I am not sure what can be done about it, though. It's a well known problem and it's wide-spread - from Amazon to Steam to the Mac App Store to even academics.

8) I'd like to see staff picks. Right now we have a problem with discovery mechanisms. It's not as bad as with the Mac App Store, but it's there. So let's fight it early. Many asset creators will tell you that you can only make real money if you are on the front page. Staff picks are one way to remedy that.

9) Asset Store competitions might improve quality. Let's say we all come to the conclusion that the Asset Store lacks in a certain area - sound for example. Then why not hold a competition on the forums or by contacting known sound asset producers - and promise the winners a fixed amount of money on top of their sales? Or some other revenue increase?

 

Yes...very good ideals that fall under the support sanity check of have self contained Owner Asset Store purchase, review, and support 'Kiosks' all in one place. We're not going to mistake an independent asset seller as Unity Technologies.

 

Sorry, I have no idea what you want to say with that.

 

The consequence of not having all the required scripts would be that you import a package and the you would get compiler errors. I don't think that this would make customers happy.

 

I like the idea of staff picks. Maybe take a step further. How about a more advanced review system?

You know, like how games are rated and reviewed by third parties. Rating for sound, art, story, originality, etc.

Could apply the same concept to asset store products. The categories would be things maybe:

1. Usability
2. Functionality
3. Code Quality
4. Documentation

10 point scale, maybe different categories for art assets. I know these things are subjective, but just throwing out idea to improve.

 

What would help discovery is that if each time an asset package was imported was if Unity reported the lines of code in error, deprecated calls, namespace errors, and of course that it was imported into a project to the Asset Owner's 'Kiosk'.

It's also help if it reported the target build platform and whether Unity was Unity Free or Unity Pro and what other assets had been imported into the same project.

 

Very simple: You don't crisscross all over town to buy a gallon of milk so why go to a half dozen different places on the internet to buy an asset and then to get support for that asset and then to refund that asset if need be? That should all be in the asset store under the Kiosk 'Goat' for assets I sold for example.

 

That's why it was suggested to import needed Unity standard asset scripts but not move them from the place were they were originally imported to by the person that built the asset if the person that build the asset did not changed the name or content of the script, e.g. Standard Assets/Water (Pro Only)/* , Editor/Water (Pro Only)/* then delete the scripting not needed when exporting the package.