Replies: 1 comment 3 replies
-
It looks like there were a couple issues with the examples I originally posted above. First, when I tried the 2nd example again today in a fresh repl, I realized it doesn't work as-is. After fixing it, I get the following error: >>> from decimal import Decimal
>>> from pint import Quantity, UnitRegistry
>>> ureg = UnitRegistry(non_int_type=Decimal)
>>> d2 = Quantity(Decimal("1.0"), ureg.inch)
>>> d2.to(ureg.thou).magnitude
Traceback (most recent call last):
[...]
TypeError: unsupported operand type(s) for ** or pow(): 'float' and 'decimal.Decimal'
>>> I think when I was originally trying this out I had accidentally typed >>> from decimal import Decimal
>>> from pint import UnitRegistry, Quantity
>>> ureg = UnitRegistry()
>>> d2 = Quantity(Decimal("1.0"), ureg.inch)
>>> d2.to(ureg.thou).magnitude
Decimal('1000.00') The second issue is that I was importing >>> from decimal import Decimal
>>> from pint import UnitRegistry
>>> ureg = UnitRegistry(non_int_type=Decimal)
>>> d2 = ureg.Quantity(Decimal("1.0"), ureg.inch)
>>> d2.to(ureg.thou).magnitude
Decimal('999.9999999999999999999999996') Couple questions:
|
Beta Was this translation helpful? Give feedback.
3 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I was surprised to find that converting
1 inch
tothou
using Decimal numbers produced a result with incorrect precision.However, if I use the Quantity() constructor, I get the correct precision I would expect using a Decimal number.
Can someone explain why these produce different results? What is the difference between creating the quantity with
Decimal("1") * ureg.inch
vs.Quantity(Decimal(1), ureg.inch)
?Beta Was this translation helpful? Give feedback.
All reactions