Custom Transform Control doesn't follow mouse

🌵 Needle Engine
1

Hello Needle,

I’m asking for help because It’s been a few days now since I’ve managed to get one of my scripts off the ground.
I’m currently creating a custom TransformControls, as I only need three rotation axes and one translation axis.
I need to be able to specifically choose the appearance of these axes.

And it also allows me to manage limitations, such as restricting certain movements and so on.

I’ve implemented almost everything (the arrows, the materials, the click and drag systems, etc).

However, I have a huge problem:
I can’t manage to make the local transformations of my object consistent with the movement of my mouse.

Whether it’s rotation or translation according to z, sometimes when I move my mouse in the direction of the arrow it goes in the other direction, sometimes not.
And my camera doesn’t move, only my object and its parents.

I tried to take my inspiration directly from TransformControl from three js but the class alone is 1600 lines long ^^'.

So I hope to find some help here. If you have a solution to guarantee this tracking that would be really great.

I’ve attached my transform creation functions, a photo and the use I make of them in my controlled object.

 const customControl = createCustomTransformControl(this.gameObject, this.context.scene, isNew);

update() {
    if(this.gizmoSelected && this.gizmoSelected.name) {
      const position = this.context.input.getPointerPosition(0)

      if(position) {
        // this.nodeComponent.mainMouthModel its parents
        // this.bevellingEnable used for move its geometry, not the object
        const value = applyGizmoTransform(this.gameObject, this.gizmoSelected.name, this.clientX - position.x, this.clientY - position.y, this.context.mainCamera, this.zValue, this.bevellingEnable, this.nodeComponent.mainMouthModel)

        this.clientX = position.x
        this.clientY = position.y
      } 
      
    }
  }

transformControlCustom.js
image.png

Original Post on Discord

by user 224464722878005248

2

Hello there – we have an experimental AI bot :robot: that might be able to help you with your question. Would you like to try this out?

How it works:
Reply to the messages of the bot that end with a thread_id to interact with it and ask follow-up questions. You can also mention the bot directly in this channel to get help.

Help requested: please wait a moment…

3

Here is what I found for you :brain: Discord

4

Hello AeistDesmos,

The issue you’re facing with inconsistent transformations relative to mouse movements is commonly due to the difference between screen space (where the mouse operates) and the object’s local or world space (where the transformations are applied). To create a consistent transformation of objects with the movement of the mouse, the following steps should help:

  1. Calculate the movement delta in screen space.
  2. Convert the delta from screen space to world space.
  3. Apply constraints if necessary (like snapping to certain angles or restricting axes).
  4. Apply the transformation in the object’s local space.

For rotations, consider using quaternions to avoid gimbal lock and integrate the rotation smoothly. The relationship between the mouse delta and the rotation angle needs to be defined. A common approach is to use the length of the mouse movement to represent the angle of rotation in radians.

For translations, the direction of the mouse movement needs to correspond to movements along the axis in world space. This usually involves projecting the mouse delta onto the translation axis and finding out the distance the object should move in that direction.

The pseudocode for rotation might look like:

// Calculate screen space delta.
const deltaX = currentMouseX - previousMouseX;
const deltaY = currentMouseY - previousMouseY;

// Convert to angle (you'll need to define the conversion factor).
const angle = Math.sqrt(deltaX * deltaX + deltaY * deltaY) * conversionFactor;

// Convert angle to world space rotation.
const axis = getRotationAxis(gizmoSelected, mainCamera);
const quaternion = new Quaternion().setFromAxisAngle(axis, angle);

// Apply constraints (if any).

// Apply rotation to object.
object.quaternion.multiplyQuaternions(quaternion, object.quaternion);

And for translation:

*thread_1IpCtgT0SsXDuvm4t8ne5ebS*
5

typescript
// Calculate screen space delta.
const deltaX = currentMouseX - previousMouseX;
const deltaY = currentMouseY - previousMouseY;

// Convert to world space direction.
const translationAxis = getTranslationAxis(gizmoSelected, mainCamera);
const worldDirection = mainCamera.getWorldDirection();
const projectedDirection = projectOntoAxis(worldDirection, translationAxis);

// Calculate movement magnitude.
const magnitude = deltaX * conversionFactor; // Change factor as needed

// Apply constraints (if any).

// Apply translation to object.
object.position.addScaledVector(projectedDirection, magnitude);

*thread_1IpCtgT0SsXDuvm4t8ne5ebS*
6

Again, the conversionFactor quantifies how much mouse movement is needed for a unit of transformation and may need to be empirically determined or adjusted based on application requirements. Constraints would include checks for restricted movements and ensuring the rotation or translation is snapped to specified increments if needed.

This is a rough outline. To implement custom gizmos and movement systems, you’ll often need a good understanding of both linear algebra and the underlying rendering engine’s capabilities. If Three.js’s TransformControls is too complex, try to build a simplified version step by step, and test each element to make sure that it operates as expected in isolation before combining them.

1 messages left GPT 4
thread_1IpCtgT0SsXDuvm4t8ne5ebS
Thanks for your feedback!

7

No body has an idea ? :c

by user 224464722878005248

8

Just arrived at work :slightly_smiling_face:
I can take a look later.

9

Generally if transforms behave in that way (flipped) then something with camera-to-world space conversion or the other way around is wrong

10

Do you only need it to function for mouse/screens/touch or also VR/AR?

11

three.js TransformControls is 1600 lines because it covers multiple cases; similar to how our DragControls are relatively complex because, well, they do a lot

12

Mouse / screen / touch, not in Vr/Ar :slightly_smiling_face:

by user 224464722878005248

13

Hi @herbst🌵
Sorry for bothering you, did you watch the problem ?
The problem is that I can’t do it with the TransformControl of three js because, in essence, it works as it is and you can’t use the translate at the same time as the rotate.
And I haven’t managed to look at how to modify Needle’s TransformControl to cover all my needs (translate and rotate at the same time, total control over guizmos meshes, limitations on transformations, geometry transformation possibility in particular).

by user 224464722878005248

14

Hello, I’m stuck actually, did somebody is available to help me ? :c
@marwie1 :cactus: ?

by user 224464722878005248

15

Have you looked into the three transform controls code for how they deal with this? Maybe you can even use the “TransformControlsPlane” class they have - otherwise you can create a regular threejs Plane and raycast against it. But i’d advice you to look into the code here first and try to figure out what they do three.js/examples/jsm/controls/TransformControls.js at d04539a76736ff500cae883d6a38b3dd8643c548 · mrdoob/three.js · GitHub

16

Hello, yeah I already saw the code, I tried to adapt it but I I didn’t succeed to reproduce it.
There are several reasons for this: the code already works in such a way that it’s impossible to combine rotation and translation at the same time.
This poses a real problem
Knowing that you’ve developed your own gizmos, which (from memory) can combine rotation and translation at the same time.
Knowing that it’s under the needle component class, which is much simpler to manage the transmission of events.

by user 224464722878005248

17

“Knowing that it’s under the needle component class, which is much simpler to manage the transmission of events.” - what do you mean here?

18

Your gizmo component is a needle component
With calls to onPointerDown, onPointerUp etc.
And mesh generation takes place within it. Knowing that it seems to me that allows you to use rotation and translation at the same time.

Three je’s transformcontrol can’t access these methods, as mesh management with raycast is done internally in a totally different way, and is not designed to manage rotation and translation at the same time.

by user 224464722878005248

19

Our TransformGizmo component also just wraps the three TransformControls - is that what you mean?

20

I thought that you used your own controls, not wrapping transformControls

So you don’t know how make custom gizmos / transformControls ?

by user 224464722878005248