Apple Pencil 2

I have a Unity painting app and I've gotten many requests to use the double tap feature on the Apple Pencil 2. I see that the new input system has pen support, and I'm attempting to use it with the code:

if(Pen.current != null)
{
if(Pen.current.tip.wasPressedThisFrame)
{
Debug.LogError("Detected Pencil Tip");
}

if(Pen.current.firstBarrelButton.wasPressedThisFrame || Pen.current.firstBarrelButton.isPressed)
{
Debug.LogError("Detected Pencil Barrel Button");
}
}

The Pencil tip log fires correctly, but not the barrel button log. Is this just not possible in the new input system yet, or am I just doing something wrong?

 

No ideas? I'd really like to use this

 

@Rene-Damm Thanks for looking into it, I have reported a bug with a simple project at https://fogbugz.unity3d.com/default.asp?1240965_b61kn2pnr0ak2cm5

 

Hi! Any progress on this?

 

Bug request was closed. Is there any news?

 

It's really disappointing that this is the top hit that comes up when looking for answers regarding being able to access input from the barrel button of the Apple Pencil 2.

The only response is "it doesn't work, file a bug and we'll get it working" followed by a filed bug that was then closed as "could not reproduce".

Was this fixed? What is the status of the Input System's iOS support? Does no one @Unity want to try and clarify what's going on?

 

@Rene-Damm Hello! Please pay attention that many devs need this feature of Apple Pencil 2. Why this bug report was closed? Your QA team was unable to reproduce it while there are several devs having this issue?

 

I've created another one issue with demo project included https://fogbugz.unity3d.com/default.asp?1381078_0llkis7pv4mnf4ja

 

Posting here too in case anyone in this thread is interested - I started writing a native plugin to add better support for the Apple Pencil and I think it might be possible to add the barrel tap fairly easily, but I only have an Apple Pencil 1 and wouldn't be able to test it. If anyone wants to collaborate on this stuff, I started a github repo here: https://github.com/all-iver/UnityApplePencil

 

Hey all_iver, I'm very interested in your work! I have a Apple Pencil 2 and want to use it for a unity app, let me know if you want some help with anything!

 

Hey, cool I am still working on my own painting app for iPad and my native plugin is a bit further along than the github link I posted. I actually have a Pencil 2 now but outside of checking that it works, I haven't had time to work on any of the newer features like barrel tap or hover. That would definitely be easier for somebody with a Mac and XCode to work on than me, since I'm just going through Unity Cloud Build from Linux. Are you mainly looking for barrel tap? Would you be interested in giving that a try (with my help)?

 

Sweet! Great to hear. Actually, I just bought a mac, but it's not coming until late December. I'd 100% be up to work on this or help! I think I want everything the Apple Pencil has to offer haha. Have you encountered any limitations using the Apple Pencil 2 with Unity?

 

To be honest I have found the Pencil 1 to be fairly quirky/difficult if you really need the full support. When you get input events, the pressure value is just an estimate because the pencil is not fast enough to give the correct pressure with the movement events. So the real pressure values come in a bit later, which can be complicated to handle because you need to go back and update the events and quickly redraw the stroke, which looks ugly if the user can see it.

To make it worse, sometimes you get a blob of very high pressure at the start of a stroke that the software never corrects, which is terrible for drawing. I've seen other people talking about that on the web but I've never seen a description of how to fix it, so I guess all the different apps are just rolling their own (imperfect) compensation.

Apple also gives you "predicted" inputs to try and reduce lag, which you also have to go back and rewrite when you get more real inputs. Using predicted events is optional, but overall supporting the Pencil fully is a fair bit more complicated than using it the way Unity is.

I think the picture is probably similar with the Pencil 2, but like I said I've only used it a bit. In my painting app, a curve with Pencil 2 on my newer iPad is noticeably less smooth than Pencil 1 on my older iPad, but I'm not sure what's causing that yet. Some smoothing algorithm is needed anyway.

Hopefully all this does not discourage you. I don't think most people should bother with estimated/predicted events, unless you're doing a drawing app where users can see the strokes, and even then just doing some kind of smoothing algorithm at 60hz resolution might be easier. It should be possible to just handle barrel tap without any of the rest of this, too.

 

The delayed pressure sounds annoying! Is this a general issue for the Apple Pencil 1?

For your initial stroke blob problem, I think I had a similar issue where you simply get the max pressure, or what's equivalent to a mouse click right? I submitted a bug exactly for this and apparently it was fixed haha. Are you using a mouse input event rather than the new input systems pen down event? Using the pen input actually fixed the problem in one of my projects.

Are you getting less samples for the Apple Pencil 2? How many samples do you get per second?

Yeah, I am more interested in achieving higher sample counts and lower latency. But I'll give it a try!

 

The delayed pressure is just how Apple Pencil works, both 1 and 2. The movement comes directly from the screen like a finger touch, but the pressure values are sent afterwards via bluetooth (I think that's how it works anyway). The initial touch event doesn't have a correct pressure value, but then you get it later from a different event handler. Unity's built-in code does not handle estimation updates.

That's interesting about the stroke blobs...I'm not so sure it is fixed though. It's not a Unity thing, it's related to the Pencil - in the past I've been able to replicate it with other apps from the app store if you turn off all smoothing and corrections. Just now I was having trouble replicating it though, naturally.

I'm using the new input system with a custom device and raw events, if you check my github repo that's how I'm still doing it. Samples should be 240/s. I have a stats view but it's aggregating the events, estimation updates and predictions so it's something like 600+ total events per second. There could definitely be a bug on my end with the Pencil 2, I just haven't looked into it much. Let's talk more on discord about it.

 

same thing with Unity politics here....
noooo support ... nothing ... it's a shame...
i'll hope and wish there will be soon a better game & ui engine...
best would be Apple only...