Skip to content

Commit 73fa02e

Browse files
committed
Add HashMapT salt, which allows creation of salt with Nat.
This allows clients to create custom salted hashmaps. For backwards compatibility we use ```haskell -- backwards compatibility type HashMap = HashMapT DefaultSalt ``` Then modify the functions to be free of salt if they can, for example insert: ```haskell insert :: forall k v salt . (Eq k, Hashable k) => k -> v -> HashMapT salt k v -> HashMapT salt k v insert k v m = insert' (hash salt k) k v m where salt = natVal (Proxy :: Proxy salt) ``` This allows the default HashMap with backwards compatibility, but also any other HashMapT I think this solves the issue with having different salts in an intersect: ```haskell intersection :: (Eq k, Hashable k) => HashMapT salt k v -> HashMapT salt k w -> HashMapT salt k v ``` Because salt is the same type variable for all arguments, it's enforced to be the same. Then you can also provide a function to resalt if the user ever ends up with different salts and still wants to do an intersect. (which results in a reconstruction of the hashmap). See thread: #319 Fix the defaultHash issues Be more verbose about default value Add comma Fix CI maybe link to source of magick number Fix default hash assertion
1 parent 6588174 commit 73fa02e

File tree

10 files changed

+437
-229
lines changed

10 files changed

+437
-229
lines changed

Data/HashMap/Internal.hs

+237-170
Large diffs are not rendered by default.

Data/HashMap/Internal/Strict.hs

+77-46
Large diffs are not rendered by default.

Data/HashMap/Lazy.hs

+6
Original file line numberDiff line numberDiff line change
@@ -28,10 +28,13 @@ module Data.HashMap.Lazy
2828
-- $strictness
2929

3030
HashMap
31+
, HashMapT
3132

3233
-- * Construction
3334
, empty
3435
, singleton
36+
, empty'
37+
, singleton'
3538

3639
-- * Basic interface
3740
, null
@@ -101,6 +104,9 @@ module Data.HashMap.Lazy
101104
, fromList
102105
, fromListWith
103106
, fromListWithKey
107+
, fromList'
108+
, fromListWith'
109+
, fromListWithKey'
104110

105111
-- ** HashSets
106112
, HS.keysSet

Data/HashMap/Strict.hs

+5-1
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,8 @@ module Data.HashMap.Strict
2626
-- * Strictness properties
2727
-- $strictness
2828

29-
HashMap
29+
HashMapT
30+
, HashMap
3031

3132
-- * Construction
3233
, empty
@@ -100,6 +101,9 @@ module Data.HashMap.Strict
100101
, fromList
101102
, fromListWith
102103
, fromListWithKey
104+
, fromList'
105+
, fromListWith'
106+
, fromListWithKey'
103107

104108
-- ** HashSets
105109
, HS.keysSet

benchmarks/Benchmarks.hs

+31-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
1-
{-# LANGUAGE CPP, DeriveAnyClass, DeriveGeneric, GADTs, PackageImports, RecordWildCards #-}
1+
{-# LANGUAGE CPP, DeriveAnyClass, DataKinds, DeriveGeneric, GADTs, PackageImports, RecordWildCards #-}
2+
3+
#if __GLASGOW_HASKELL__ >= 802
4+
{-# LANGUAGE TypeApplications #-}
5+
#endif
26

37
module Main where
48

@@ -342,11 +346,30 @@ main = do
342346
, bench "ByteString" $ whnf HM.fromList elemsBS
343347
, bench "Int" $ whnf HM.fromList elemsI
344348
]
349+
#if __GLASGOW_HASKELL__ >= 802
350+
, bgroup "long custom salt" -- 18446744073710551615 = fromInteger ((toInteger (maxBound :: Word64)) + 1000000)
351+
[ bench "String" $ whnf (HM.fromList' @_ @_ @18446744073710551615) elems
352+
, bench "ByteString" $ whnf (HM.fromList' @_ @_ @18446744073710551615) elemsBS
353+
, bench "Int" $ whnf (HM.fromList' @_ @_ @18446744073710551615) elemsI
354+
]
355+
#endif
345356
, bgroup "short"
346357
[ bench "String" $ whnf HM.fromList elemsDup
347358
, bench "ByteString" $ whnf HM.fromList elemsDupBS
348359
, bench "Int" $ whnf HM.fromList elemsDupI
349360
]
361+
#if __GLASGOW_HASKELL__ >= 802
362+
, bgroup "short custom salt" -- 18446744073710551615 * 10
363+
[ bench "String" $ whnf (HM.fromList' @_ @_ @184467440737105516150) elemsDup
364+
, bench "ByteString" $ whnf (HM.fromList' @_ @_ @184467440737105516150) elemsDupBS
365+
, bench "Int" $ whnf (HM.fromList' @_ @_ @184467440737105516150) elemsDupI
366+
]
367+
, bgroup "short custom salt 42" -- 18446744073710551615 * 10
368+
[ bench "String" $ whnf (HM.fromList' @_ @_ @42) elemsDup
369+
, bench "ByteString" $ whnf (HM.fromList' @_ @_ @42) elemsDupBS
370+
, bench "Int" $ whnf (HM.fromList' @_ @_ @42) elemsDupI
371+
]
372+
#endif
350373
]
351374
-- fromListWith
352375
, bgroup "fromListWith"
@@ -360,6 +383,13 @@ main = do
360383
, bench "ByteString" $ whnf (HM.fromListWith (+)) elemsDupBS
361384
, bench "Int" $ whnf (HM.fromListWith (+)) elemsDupI
362385
]
386+
#if __GLASGOW_HASKELL__ >= 802
387+
, bgroup "short custom salt"
388+
[ bench "String" $ whnf ((HM.fromListWith' @_ @_ @10) (+)) elemsDup
389+
, bench "ByteString" $ whnf ((HM.fromListWith' @_ @_ @10) (+)) elemsDupBS
390+
, bench "Int" $ whnf ((HM.fromListWith' @_ @_ @10) (+)) elemsDupI
391+
]
392+
#endif
363393
]
364394
]
365395
]

docs/migration-salt.md

+33
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
The salt changes are backwards compatible.
2+
3+
However, if you want to let a client make use of
4+
custom salts some effort is required,
5+
running commands like these should get you somewhere:
6+
7+
```ed
8+
:%s/HashMap k/HashMapT salt k/g
9+
:%s/Hashable k)/Hashable k, KnownNat salt)
10+
```
11+
12+
HashMap is now an alias to HashMapT with a hardcoded DefaultSalt.
13+
These changes allow salt to be anything.
14+
semigroup operations (a -> a -> a) can use the same salt to guarantee
15+
not having to rebuild the hahsmap.
16+
17+
18+
If you encounter this error:
19+
```
20+
• Illegal instance declaration for ‘SomeTypeClass (HashMap k v)’
21+
(All instance types must be of the form (T t1 ... tn)
22+
where T is not a synonym.
23+
Use TypeSynonymInstances if you want to disable this.)
24+
```
25+
usually it's good enough to provide the instance with a free salt:
26+
27+
```haskell
28+
instance SomeTypeClass (HashMap salt k v) where
29+
...
30+
31+
```
32+
If it it starts complaining about not salt not matching DefaultSalt,
33+
use the `'` constructors such as `empty'` and `singleton'`

tests/HashMapProperties.hs

+6-4
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ module Main (main) where
88

99
import Control.Monad ( guard )
1010
import qualified Data.Foldable as Foldable
11+
import GHC.TypeLits(KnownNat)
1112
#if MIN_VERSION_base(4,10,0)
1213
import Data.Bifoldable
1314
#endif
@@ -16,11 +17,11 @@ import Data.Hashable (Hashable(hashWithSalt))
1617
import qualified Data.List as L
1718
import Data.Ord (comparing)
1819
#if defined(STRICT)
19-
import Data.HashMap.Strict (HashMap)
20+
import Data.HashMap.Strict (HashMap, HashMapT)
2021
import qualified Data.HashMap.Strict as HM
2122
import qualified Data.Map.Strict as M
2223
#else
23-
import Data.HashMap.Lazy (HashMap)
24+
import Data.HashMap.Lazy (HashMap, HashMapT)
2425
import qualified Data.HashMap.Lazy as HM
2526
import qualified Data.Map.Lazy as M
2627
#endif
@@ -41,8 +42,9 @@ newtype Key = K { unK :: Int }
4142
instance Hashable Key where
4243
hashWithSalt salt k = hashWithSalt salt (unK k) `mod` 20
4344

44-
instance (Eq k, Hashable k, Arbitrary k, Arbitrary v) => Arbitrary (HashMap k v) where
45-
arbitrary = fmap (HM.fromList) arbitrary
45+
instance (Eq k, Hashable k, Arbitrary k, Arbitrary v, KnownNat salt)
46+
=> Arbitrary (HashMapT salt k v) where
47+
arbitrary = fmap (HM.fromList') arbitrary
4648

4749
------------------------------------------------------------------------
4850
-- * Properties

tests/Regressions.hs

+31-1
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,15 @@
11
{-# LANGUAGE ScopedTypeVariables #-}
22
{-# LANGUAGE MagicHash #-}
33
{-# LANGUAGE UnboxedTuples #-}
4+
{-# LANGUAGE CPP #-}
5+
46
module Main where
57

68
import Control.Applicative ((<$>))
79
import Control.Exception (evaluate)
810
import Control.Monad (replicateM)
911
import Data.Hashable (Hashable(..))
12+
import qualified Data.Hashable as HAS
1013
import qualified Data.HashMap.Strict as HM
1114
import qualified Data.HashMap.Lazy as HML
1215
import Data.List (delete)
@@ -16,11 +19,14 @@ import GHC.IO (IO (..))
1619
import System.Mem (performGC)
1720
import System.Mem.Weak (mkWeakPtr, deRefWeak)
1821
import System.Random (randomIO)
19-
import Test.HUnit (Assertion, assert)
22+
import Test.HUnit (Assertion, assert, (@=?))
2023
import Test.Framework (Test, defaultMain)
2124
import Test.Framework.Providers.HUnit (testCase)
2225
import Test.Framework.Providers.QuickCheck2 (testProperty)
2326
import Test.QuickCheck
27+
import Data.Proxy
28+
import qualified Data.HashMap.Internal as Internal
29+
import Data.Text(Text, pack)
2430

2531
issue32 :: Assertion
2632
issue32 = assert $ isJust $ HM.lookup 7 m'
@@ -124,6 +130,28 @@ issue254Strict = do
124130
touch mp
125131
assert $ isNothing res
126132

133+
goldenHash :: Assertion
134+
goldenHash = do
135+
HAS.hash someString @=?
136+
fromIntegral (toInteger (Internal.hash (Proxy :: Proxy Internal.DefaultSalt) someString))
137+
138+
someString :: Text
139+
someString = pack "hello world"
140+
141+
assertDefaultHash :: Assertion
142+
assertDefaultHash = do
143+
-- I only found this by testing, the WORD_SIZE_IN_BITS doesn't
144+
-- appear to be set, we get the warning, yet the
145+
-- larger value in upstream is still being used.
146+
-- https://github.com/haskell-unordered-containers/hashable/blob/master/src/Data/Hashable/Class.hs#L221
147+
#if MIN_VERSION_hashable(1,3,1)
148+
HAS.hash someString @=?
149+
HAS.hashWithSalt 14695981039346656037 someString
150+
#else
151+
HAS.hash someString @=?
152+
HAS.hashWithSalt 0xdc36d1615b7400a4 someString
153+
#endif
154+
127155
------------------------------------------------------------------------
128156
-- * Test list
129157

@@ -135,6 +163,8 @@ tests =
135163
, testProperty "issue39b" propEqAfterDelete
136164
, testCase "issue254 lazy" issue254Lazy
137165
, testCase "issue254 strict" issue254Strict
166+
, testCase "make sure default hash remains the same for backwards compatbility" goldenHash
167+
, testCase "asserts the default hash in case they change it" assertDefaultHash
138168
]
139169

140170
------------------------------------------------------------------------

tests/Strictness.hs

+8-5
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
{-# LANGUAGE CPP, FlexibleInstances, GeneralizedNewtypeDeriving #-}
2+
{-# LANGUAGE ScopedTypeVariables #-}
23
{-# OPTIONS_GHC -fno-warn-orphans #-}
34

45
module Main (main) where
@@ -14,13 +15,15 @@ import Data.Maybe (fromMaybe, isJust)
1415
import Control.Arrow (second)
1516
import Control.Monad (guard)
1617
import Data.Foldable (foldl')
18+
import GHC.TypeLits(KnownNat)
1719
#if !MIN_VERSION_base(4,8,0)
1820
import Data.Functor ((<$))
1921
import Data.Foldable (all)
2022
import Prelude hiding (all)
2123
#endif
2224

23-
import Data.HashMap.Strict (HashMap)
25+
26+
import Data.HashMap.Strict (HashMapT, HashMap)
2427
import qualified Data.HashMap.Strict as HM
2528

2629
-- Key type that generates more hash collisions.
@@ -30,9 +33,9 @@ newtype Key = K { unK :: Int }
3033
instance Hashable Key where
3134
hashWithSalt salt k = hashWithSalt salt (unK k) `mod` 20
3235

33-
instance (Arbitrary k, Arbitrary v, Eq k, Hashable k) =>
34-
Arbitrary (HashMap k v) where
35-
arbitrary = HM.fromList `fmap` arbitrary
36+
instance (Arbitrary k, Arbitrary v, Eq k, Hashable k, KnownNat salt) =>
37+
Arbitrary (HashMapT salt k v) where
38+
arbitrary = HM.fromList' `fmap` arbitrary
3639

3740
instance Show (Int -> Int) where
3841
show _ = "<function>"
@@ -100,7 +103,7 @@ pFromListWithKeyStrict f =
100103
-- could be lazy in the "new" value. fromListWith must, however,
101104
-- be strict in whatever value is actually inserted into the map.
102105
-- Getting all these properties specified efficiently seems tricky.
103-
-- Since it's not hard, we verify that the converted HashMap has
106+
-- Since it's not hard, we verify that the converted HashMapT salt has
104107
-- no unforced values. Rather than trying to go into detail for the
105108
-- rest, this test compares the strictness behavior of fromListWith
106109
-- to that of insertWith. The latter should be easier to specify

unordered-containers.cabal

+3-1
Original file line numberDiff line numberDiff line change
@@ -158,7 +158,9 @@ test-suite regressions
158158
test-framework >= 0.3.3,
159159
test-framework-hunit,
160160
test-framework-quickcheck2,
161-
unordered-containers
161+
unordered-containers,
162+
text,
163+
hashable
162164

163165
default-language: Haskell2010
164166
ghc-options: -Wall

0 commit comments

Comments
 (0)