diff --git a/.gitignore b/.gitignore index 4445c82..c368d45 100644 --- a/.gitignore +++ b/.gitignore @@ -1,4 +1,2 @@ -dist -.cabal-sandbox -cabal.sandbox.config -src/**/.* +.stack-work/ +*~ \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..bb4e4e3 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,11 @@ +# Changelog for `sgCFG` + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), +and this project adheres to the +[Haskell Package Versioning Policy](https://pvp.haskell.org/). + +## Unreleased + +## 0.1.0.0 - YYYY-MM-DD diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..98b6ab8 --- /dev/null +++ b/LICENSE @@ -0,0 +1,30 @@ +Copyright Samuel Gfrörer (c) 2024 + +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials provided + with the distribution. + + * Neither the name of Author name here nor the names of other + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/README.md b/README.md new file mode 100644 index 0000000..06eb5ed --- /dev/null +++ b/README.md @@ -0,0 +1,113 @@ +# sgCFG + +## Description + +sgCFG is a command line tool for processing context-free grammars, in particular applying transformations to a grammar. +A program can read and write grammars in various formats. +Also the details of the format can be modified via command line arguments. + +## Example Use Case + +There are many strategies for writing parsers based on a given context-free grammar. +In functional languages *parser combinators* such as [parsec](https://hackage.haskell.org/package/parsec) are a common and convenient approach, since the code for a parser may be (more or less) directly derived from a given grammar, maybe even automatically generated. +In such scenarios, the shape of the input grammar has an important influence on the effiency and even on termination properties of a parser. +One might find the need to transform or "optimise" a grammar beforehand (usually without changing the language). +This tool may be used to apply such transformations. + +## Notable Features + +- read/write a grammar in backus-naur format +- read/write a grammar using grouped format(A -> X1 | ... | Xn) or ungrouped format (A -> X1, ... , A -> Xn) +- apply transformations to a grammar, notably: + + - left factoring + - left recursion elimination + - elimination of epsilon rules + - isolation of the grammar for a specific nonterminal + - finding unused rules + - calculation of "FIRST" sets as described in the famous [Dragonbook](#bibliography) + +## Remark + +This software is work in progress. Bug reports and pull requests are welcome. + +## Prerequisits + +- [stack](https://haskellstack.org) + +## Build + + $ stack build + +## Installation + +The executable can be run from a local directory: + + $ stack exec -- sgCFG [ARGS...] + +To be able to invoke sgCFG from the command line without having to type the whole path to the executable, issue + + $ stack install + +and make sure that `stack path --local-bin` is in your `PATH`. Now, the program may be run from the command line: + + $ sgCFG [ARGS...] + +## Usage + +(this passage assumes that the sgCFG executable is in your PATH) + +To print a list of OPTIONS, run: + + $ sgCFG -h + +The program is implemented after the well known "filter" paradigm used for many linux/unix utilites - it reads a grammar from stdin, and outputs a grammar to stdout (potentially after applying some transformations to it). +Also, it logs what is being done to stderr. + +If you have a text file "file" containing your grammar, you can use sgCFG like this: + + $ cat "file" | sgCFG --input-format=default --output-grouped=default [OPTIONS] 2>"logfile.log" > "outputfile" + +*Please notice*: the options "--input-format" and "--output" (or "--output-grouped") are always needed! + +## Examples + +- Identity (no transformations), *default* output format + + $ echo -e 'A -> A "a" | "a"' | stack exec -- sgCFG -i default -d default + A-> + A "a" + A-> + "a" + +- Identity (no transformations), *grouped* output format + + $ echo -e 'A -> A "a" | "a"' | stack exec -- sgCFG -i default -g default + A -> + A "a" + |"a" + +- Eliminate Left Recursion + + $ echo -e 'A -> A "a" | "a"' | stack exec -- sgCFG -i default -g default -t 'elimLeftRec(%v%n)' 2>/dev/null + A -> + "a" A0 + A0 -> + "a" A0 + |"" + +- Chaining multiple transformations: Eliminate Left Recursion > Eliminate Epsilon Rules: + + $ echo -e 'A -> A "a" | "a"' | stack exec -- sgCFG -i default -g default -t 'elimLeftRec(%v%n)' -t 'elimEpsilon()' 2>/dev/null + A -> + "a" A0 + |"a" + A0 -> + "a" A0 + |"a + +## Bibliography + +[Dragonbook] + Alfred V. Aho, Monica S. Lam, Ravi Sethi, and Jeffrey D. Ullman: + Compilers: Principles, Techniques, and Tools diff --git a/Setup.hs b/Setup.hs new file mode 100644 index 0000000..9a994af --- /dev/null +++ b/Setup.hs @@ -0,0 +1,2 @@ +import Distribution.Simple +main = defaultMain diff --git a/src/ConfigFromArgs.hs b/app/ConfigFromArgs.hs similarity index 88% rename from src/ConfigFromArgs.hs rename to app/ConfigFromArgs.hs index 71a0a69..7731e0f 100644 --- a/src/ConfigFromArgs.hs +++ b/app/ConfigFromArgs.hs @@ -7,7 +7,6 @@ module ConfigFromArgs( ) where import Config.Types ---import Grammar import GrammarFormat import Types import Utils (mapSnd) @@ -15,9 +14,7 @@ import Utils (mapSnd) import qualified System.Console.GetOpt as Opt import Control.Monad import Control.Monad.Except ---import Text.Read ---import Control.Applicative configFromArgs :: MonadError String m => [String] -> m Config configFromArgs args = @@ -26,11 +23,9 @@ configFromArgs args = let f = foldl (>=>) return options in - -- maybe (throwError "error parsing options") return $ - f $ defConfig + f $ defConfig (_,_,errMessages) -> throwError $ "option format error: " ++ unlines errMessages - -- Nothing --errMessages usageString :: String -> String usageString progName = @@ -100,8 +95,8 @@ optDescrList = Opt.ReqArg transformation "GRAMMAR_TRANSFORMATION" ] where - defineOption shortOpt longOpt descr transformation = - Opt.Option shortOpt longOpt transformation descr + defineOption shortOpt longOpt descr transformation' = + Opt.Option shortOpt longOpt transformation' descr outputTree :: MonadError String m => Config -> m Config outputTree = @@ -111,9 +106,13 @@ outputTree = case spec of OutputGroupedGrammar info -> return $ OutputGroupedGrammar $ info{ outputGrammar_asTree = True } - _ -> --Nothing + _ -> throwError $ "bla" +mapToHeadOrError' :: + MonadError p m => + p -> (a -> m a) + -> [a] -> m [a] mapToHeadOrError' errMsg f list = case list of [] -> throwError $ errMsg @@ -121,7 +120,6 @@ mapToHeadOrError' errMsg f list = new <- f x return $ new:xs - inputF :: MonadError String m => String -> Config -> m Config inputF arg cfg = do @@ -151,6 +149,9 @@ changeInputFormat arg cfg = do (key, val) <- either throwError return $ parseKeyValue arg cfgMapToInputFormatM (applyFormatChange key val) cfg +parseKeyValue :: + FromPretty a => + [Char] -> Either ParseError (a, [Char]) parseKeyValue str = case mapSnd (drop 1) $ span (/='=') str of (keyStr, val) -> do @@ -172,9 +173,9 @@ transformation arg = =<< outputGrammarInfo_mapToTransformationsM (changeTransformations str) info where - changeTransformations str list = do - transformation <- either throwError return $ fromPretty str - return $ (list++[transformation]) + changeTransformations str' list = do + transformation' <- either throwError return $ fromPretty str' + return $ (list++[transformation']) _ -> throwError $ "not defined for grouped grammars yet!" @@ -187,19 +188,15 @@ changeOutputFormat arg cfg = where changeF :: FormatParam -> String -> [OutputSpec] -> m [OutputSpec] changeF key val outputCommands = - --maybe (throwError "outputCommands empty") return $ flip (mapToHeadOrError' "") outputCommands $ \spec -> - --(maybe (throwError "") return . flip mapToHeadMaybe outputCommands) =<< \spec -> case spec of OutputGrammar info -> - --return $ - fmap OutputGrammar $ - outputGrammarInfo_mapToFormatM (applyFormatChange key val) info + fmap OutputGrammar $ + outputGrammarInfo_mapToFormatM (applyFormatChange key val) info OutputGroupedGrammar info -> - --return $ - fmap OutputGroupedGrammar $ - outputGrammarInfo_mapToFormatM (applyFormatChange key val) info + fmap OutputGroupedGrammar $ + outputGrammarInfo_mapToFormatM (applyFormatChange key val) info _ -> throwError $ "bla" @@ -240,10 +237,5 @@ applyFormatChange param str outputInfo FormatState (f alreadySet format) (if alreadySet then overwrittenParams else param:overwrittenParams) - {- - case param `elem` overwrittenParams of - False -> FormatState (f (const [str]) format) (param:overwrittenParams) - True -> FormatState (f (str:) format) overwrittenParams - -} format = formatState_format outputInfo overwrittenParams = formatState_paramsChanged outputInfo diff --git a/src/Main.hs b/app/Main.hs similarity index 91% rename from src/Main.hs rename to app/Main.hs index ef0d34e..58dde0a 100644 --- a/src/Main.hs +++ b/app/Main.hs @@ -1,10 +1,11 @@ -module Main where +module Main (main) where + +import ConfigFromArgs import Parse.Token import Grammar import GroupedGrammar import Types -import ConfigFromArgs import Parse.ParseFormatFromGrammarFormat (parseFormatFromGrammarFormat) import Utils.Logging @@ -12,9 +13,8 @@ import Utils.Logging import qualified System.Environment as Env (getArgs, getProgName) import Control.Monad import Control.Monad.Except ---import Control.Monad.Writer -import Data.List ---import Data.Maybe(fromMaybe) +import Control.Monad.Trans +import Data.List( intercalate ) main :: IO () @@ -97,6 +97,9 @@ unpackTransformationMonad e = --exit Right x -> return x +tokStreamToText :: + (Foldable t, HasTokenType a, Pretty a) => + t a -> [Char] tokStreamToText s = intercalate " " $ foldl conc [] $ @@ -107,11 +110,3 @@ tokStreamToText s = case tokenType next of SepTokenType -> [pretty next] ++ ["\n"] _ -> [pretty next] - -{- -failOrVal :: Monad m => String -> Either String a -> m a -failOrVal msg x = - case x of - Left err -> fail $ msg ++ err - Right val -> return val --} diff --git a/package.yaml b/package.yaml new file mode 100644 index 0000000..733ab9e --- /dev/null +++ b/package.yaml @@ -0,0 +1,64 @@ +name: sgCFG +version: 0.1.0.0 +github: "EsGeh/sgCFG" +license: BSD-3-Clause +author: "EsGeh" +maintainer: "samuel@esgeh.net" +copyright: "2024 Samuel Gfrörer" + +extra-source-files: +- README.md +- CHANGELOG.md + +# Metadata used when publishing your package +synopsis: "command line tool for processing context-free grammarars" +category: "Formal Languages" + +# To avoid duplicated efforts in documentation and dealing with the +# complications of embedding Haddock markup inside cabal files, it is +# common to point users to the README.md file. +description: Please see the README on GitHub at + +dependencies: +- base >= 4.7 && < 5 +- parsec == 3.1.* +- mtl == 2.3.* +- containers +- regex-tdfa +- regex-base + +ghc-options: +- -Wall +- -Wcompat +- -Widentities +- -Wincomplete-record-updates +- -Wincomplete-uni-patterns +- -Wmissing-home-modules +- -Wpartial-fields +- -Wredundant-constraints +- -fno-warn-tabs + +library: + source-dirs: src + +executables: + sgCFG: + main: Main.hs + source-dirs: app + ghc-options: + - -threaded + - -rtsopts + - -with-rtsopts=-N + dependencies: + - sgCFG + +tests: + sgCFG-test: + main: Spec.hs + source-dirs: test + ghc-options: + - -threaded + - -rtsopts + - -with-rtsopts=-N + dependencies: + - sgCFG diff --git a/sgCFG.cabal b/sgCFG.cabal index 4849c93..d06b06d 100644 --- a/sgCFG.cabal +++ b/sgCFG.cabal @@ -1,31 +1,118 @@ --- Initial grammarAnalizer.cabal generated by cabal init. For further --- documentation, see http://haskell.org/cabal/users-guide/ +cabal-version: 2.2 -name: sgCFG -version: 0.1.0.0 --- synopsis: --- description: --- license: -license-file: LICENSE -author: EsGeh -maintainer: SamuelGfroerer@googlemail.com --- copyright: -category: Language -build-type: Simple --- extra-source-files: -cabal-version: >=1.10 +-- This file has been generated from package.yaml by hpack version 0.35.2. +-- +-- see: https://github.com/sol/hpack + +name: sgCFG +version: 0.1.0.0 +synopsis: command line tool for processing context-free grammarars +description: Please see the README on GitHub at +category: Formal Languages +homepage: https://github.com/EsGeh/sgCFG#readme +bug-reports: https://github.com/EsGeh/sgCFG/issues +author: EsGeh +maintainer: samuel@esgeh.net +copyright: 2024 Samuel Gfrörer +license: BSD-3-Clause +license-file: LICENSE +build-type: Simple +extra-source-files: + README.md + CHANGELOG.md + +source-repository head + type: git + location: https://github.com/EsGeh/sgCFG + +library + exposed-modules: + Config.Types + Grammar + Grammar.Types + GrammarFormat + GroupedGrammar + GroupedGrammar.Conversions + GroupedGrammar.Parse + GroupedGrammar.Transformations + GroupedGrammar.Transformations.AddActionSymbols + GroupedGrammar.Transformations.AddProds + GroupedGrammar.Transformations.BreakProds + GroupedGrammar.Transformations.DeleteProds + GroupedGrammar.Transformations.ElimEpsilon + GroupedGrammar.Transformations.ElimLeftRecur + GroupedGrammar.Transformations.FindDeadEnds + GroupedGrammar.Transformations.FindLoops + GroupedGrammar.Transformations.FirstSet + GroupedGrammar.Transformations.LeftFactor + GroupedGrammar.Transformations.RemoveDoubleProds + GroupedGrammar.Transformations.Types + GroupedGrammar.Transformations.Unfold + GroupedGrammar.Transformations.Utils + GroupedGrammar.Transformations.VarNameMonad + GroupedGrammar.Types + Lib + Parse.Format + Parse.ParseFormatFromGrammarFormat + Parse.Token + Parse.Token.Internals + Parse.Token.Parse + Types + Utils + Utils.Graph + Utils.Logging + other-modules: + Paths_sgCFG + autogen-modules: + Paths_sgCFG + hs-source-dirs: + src + ghc-options: -Wall -Wcompat -Widentities -Wincomplete-record-updates -Wincomplete-uni-patterns -Wmissing-home-modules -Wpartial-fields -Wredundant-constraints -fno-warn-tabs + build-depends: + base >=4.7 && <5 + , containers + , mtl ==2.3.* + , parsec ==3.1.* + , regex-base + , regex-tdfa + default-language: Haskell2010 executable sgCFG - main-is: Main.hs - -- other-modules: - -- other-extensions: + main-is: Main.hs + other-modules: + ConfigFromArgs + Paths_sgCFG + autogen-modules: + Paths_sgCFG + hs-source-dirs: + app + ghc-options: -Wall -Wcompat -Widentities -Wincomplete-record-updates -Wincomplete-uni-patterns -Wmissing-home-modules -Wpartial-fields -Wredundant-constraints -fno-warn-tabs -threaded -rtsopts -with-rtsopts=-N + build-depends: + base >=4.7 && <5 + , containers + , mtl ==2.3.* + , parsec ==3.1.* + , regex-base + , regex-tdfa + , sgCFG + default-language: Haskell2010 + +test-suite sgCFG-test + type: exitcode-stdio-1.0 + main-is: Spec.hs + other-modules: + Paths_sgCFG + autogen-modules: + Paths_sgCFG + hs-source-dirs: + test + ghc-options: -Wall -Wcompat -Widentities -Wincomplete-record-updates -Wincomplete-uni-patterns -Wmissing-home-modules -Wpartial-fields -Wredundant-constraints -fno-warn-tabs -threaded -rtsopts -with-rtsopts=-N build-depends: - base == 4.*, - parsec == 3.1.*, - regex-tdfa, - regex-base, - mtl == 2.2.*, - containers - hs-source-dirs: src - default-language: Haskell2010 - ghc-options: -W -fno-warn-tabs + base >=4.7 && <5 + , containers + , mtl ==2.3.* + , parsec ==3.1.* + , regex-base + , regex-tdfa + , sgCFG + default-language: Haskell2010 diff --git a/src/Config/Types.hs b/src/Config/Types.hs index b21ebfe..678ab64 100644 --- a/src/Config/Types.hs +++ b/src/Config/Types.hs @@ -1,11 +1,10 @@ {-# LANGUAGE ScopedTypeVariables #-} +{-# LANGUAGE FlexibleContexts #-} module Config.Types where import Grammar.Types import GrammarFormat import GroupedGrammar ---import GroupedGrammar.Transformations ---import GroupedGrammar.Types import Types import Utils @@ -17,20 +16,35 @@ data Config = Config { cfg_inputFormat :: FormatState, cfg_output :: [OutputSpec] - --cfg_outputFormat :: GrammarFormat } deriving (Show) + +cfgMapToInputFormatM :: + Monad m => + (FormatState -> m FormatState) -> Config -> m Config cfgMapToInputFormatM f cfg = do new <- f (cfg_inputFormat cfg) return $ cfg{ cfg_inputFormat = new } ---cfgMapToInputFormat f cfg = cfg{ cfg_inputFormat = f (cfg_inputFormat cfg) } + +cfgMapToOutputM :: + Monad m => + ([OutputSpec] -> m [OutputSpec]) + -> Config -> m Config cfgMapToOutputM f cfg = do new <- f (cfg_output cfg) return $ cfg{ cfg_output = new } ---cfgMapToOutput f cfg = cfg{ cfg_output = f (cfg_output cfg) } +cfgMapToInputFormat :: + (FormatState -> FormatState) + -> Config -> Config cfgMapToInputFormat f = runIdentity . cfgMapToInputFormatM (return . f) + +cfgMapToOutput :: + ([OutputSpec] -> [OutputSpec]) + -> Config -> Config cfgMapToOutput f = runIdentity . cfgMapToOutputM (return . f) + +defConfig :: Config defConfig = Config { cfg_inputFormat = FormatState (defaultFormat Default) [], @@ -52,19 +66,36 @@ data OutputGrammarInfo outputGrammar_asTree :: Bool } deriving (Show) + +defOutputGrammarInfo :: DefaultFormat -> OutputGrammarInfo defOutputGrammarInfo f = OutputGrammarInfo (defFormatState f) [] False + +outputGrammarInfo_mapToFormatM :: + Monad m => + (FormatState -> m FormatState) + -> OutputGrammarInfo -> m OutputGrammarInfo outputGrammarInfo_mapToFormatM f x = do new <- f $ outputGrammar_format x return $ x{ outputGrammar_format = new } + +outputGrammarInfo_mapToTransformationsM :: + Monad m => + ([Transformation] -> m [Transformation]) + -> OutputGrammarInfo -> m OutputGrammarInfo outputGrammarInfo_mapToTransformationsM f x = do new <- f $ outputGrammar_transformations x return $ x{ outputGrammar_transformations = new } + +outputGrammarInfo_mapToFormat :: + (FormatState -> FormatState) + -> OutputGrammarInfo -> OutputGrammarInfo outputGrammarInfo_mapToFormat f = runIdentity . outputGrammarInfo_mapToFormatM (return . f) + +outputGrammarInfo_mapToTransformations :: + ([Transformation] -> [Transformation]) + -> OutputGrammarInfo -> OutputGrammarInfo outputGrammarInfo_mapToTransformations f = runIdentity . outputGrammarInfo_mapToTransformationsM (return . f) -{- -outputGrammarInfo_mapToFormat f x = x{ outputGrammar_format = f (outputGrammar_format x) } -outputGrammarInfo_mapToTransformations f x = x{ outputGrammar_transformations = f (outputGrammar_transformations x) } --} + data FormatState = FormatState { @@ -73,10 +104,21 @@ data FormatState formatState_paramsChanged :: [FormatParam] } deriving (Show) + +defFormatState :: DefaultFormat -> FormatState defFormatState f = FormatState (defaultFormat f) [] + +formatState_mapToFormat :: + (GrammarFormat -> GrammarFormat) + -> FormatState -> FormatState formatState_mapToFormat f x = x{ formatState_format = f (formatState_format x) } + +formatState_mapToParamsChanged :: + ([FormatParam] -> [FormatParam]) + -> FormatState -> FormatState formatState_mapToParamsChanged f x = x{ formatState_paramsChanged = f (formatState_paramsChanged x) } + data FormatParam = LeftSideFormatParam | RightSideFormatParam @@ -88,6 +130,7 @@ data FormatParam deriving (Eq, Show) instance FromPretty Transformation where + -- fromPretty :: String -> Either ParseError Transformation fromPretty str = case parseTransformationDescr str of ("annotate", ["loops"]) -> @@ -96,12 +139,12 @@ instance FromPretty Transformation where return $ Annotate $ AnnotateWithFirstSet ("leftFactor", [varScheme]) -> fmap LeftFactor $ fromPretty varScheme - ("leftFactor_full", [whileCond, negate, regex, varScheme]) -> + ("leftFactor_full", [whileCond, negate', regex, varScheme]) -> LeftFactor_Full <$> fromPretty whileCond <*> ( VarCondition <$> - isEqualOrEmpty "not" negate + isEqualOrEmpty "not" negate' <*> return regex ) <*> fromPretty varScheme @@ -109,29 +152,29 @@ instance FromPretty Transformation where fmap ElimLeftRecur $ fromPretty varScheme ("elimLeftRec_noEpsilon", [varScheme]) -> fmap ElimLeftRecurNoEpsilon $ fromPretty varScheme - ("elimLeftRec_full", [negate,regex,varScheme]) -> + ("elimLeftRec_full", [negate',regex,varScheme]) -> ElimLeftRecur_Full <$> ( VarCondition <$> - isEqualOrEmpty "not" negate + isEqualOrEmpty "not" negate' <*> return regex ) <*> fromPretty varScheme - ("elimLeftRec_noEpsilon_full", [negate,regex,varScheme]) -> + ("elimLeftRec_noEpsilon_full", [negate',regex,varScheme]) -> ElimLeftRecurNoEpsilon_Full <$> ( VarCondition <$> - isEqualOrEmpty "not" negate + isEqualOrEmpty "not" negate' <*> return regex ) <*> fromPretty varScheme ("breakRules", [maxLength, varScheme]) -> fmap (uncurry BreakRules) $ liftM2 (,) (return $ read maxLength) (fromPretty varScheme) - ("unfold", [negate,regex]) -> + ("unfold", [negate',regex]) -> do - doNegate <- isEqualOrEmpty "not" negate + doNegate <- isEqualOrEmpty "not" negate' return $ Unfold $ VarCondition { varCond_negate = doNegate, @@ -158,9 +201,9 @@ instance FromPretty Transformation where insertProdsParams_position = pos, insertProdsParams_productions = prods } - ("delete", [negate,regex]) -> + ("delete", [negate',regex]) -> do - doNegate <- isEqualOrEmpty "not" negate + doNegate <- isEqualOrEmpty "not" negate' return $ DeleteProductions $ VarCondition { @@ -185,11 +228,12 @@ instance FromPretty Transformation where , show x ] +isEqualOrEmpty :: MonadError String m => String -> String -> m Bool isEqualOrEmpty pattern str = case str of str' | str'==pattern -> return $ True "" -> return $ False - _ -> fail $ + _ -> throwError $ concat $ [ "isEqualOrEmpty failed with " , pattern @@ -251,12 +295,14 @@ instance FromPretty FormatParam where "lineComment" -> return LineCommentFormatParam _ -> Left $ concat $ ["unknown FormatParam", "\"", str, "\""] +mapToHeadMaybe :: (a -> a) -> [a] -> Maybe [a] mapToHeadMaybe f list = case list of [] -> Nothing (x:xs) -> do return $ (f x):xs +mapToHeadOrError :: MonadError p m => p -> (a -> a) -> [a] -> m [a] mapToHeadOrError errMsg f list = case list of [] -> throwError $ errMsg diff --git a/src/Grammar.hs b/src/Grammar.hs index 26974d8..a5a16d9 100644 --- a/src/Grammar.hs +++ b/src/Grammar.hs @@ -16,6 +16,8 @@ import Grammar.Types import Types import GrammarFormat import GroupedGrammar +import Parse.Token +import Parse.Format import Parse.ParseFormatFromGrammarFormat (parseFormatFromGrammarFormat) import Utils (mapLeft) @@ -26,9 +28,15 @@ instance FromTextAs GrammarFormat Grammar where in mapLeft show $ grammarFromStr parseFormat str +grammarFromStr :: + ParseFormat -> String + -> Either String (GrammarGen Production) grammarFromStr descr = fmap grammarFromGroupedGrammar . groupedGrammarFromStr descr +grammarFromTokens :: + [Token] + -> Either String (GrammarGen Production) grammarFromTokens = fmap grammarFromGroupedGrammar . groupedGrammarFromTokens diff --git a/src/Grammar/Types.hs b/src/Grammar/Types.hs index a3f2158..f499d49 100644 --- a/src/Grammar/Types.hs +++ b/src/Grammar/Types.hs @@ -23,10 +23,14 @@ data GrammarGen prod = } deriving (Eq, Ord, Show, Functor, Fold.Foldable, Traversable) +grammar_mapToProductionsM :: Monad m => + ([prod1] -> m [prod2]) + -> GrammarGen prod1 -> m (GrammarGen prod2) grammar_mapToProductionsM f g = do new <- f $ fromGrammar g return $ Grammar $ new +grammar_mapToProductions :: ([prod1] -> [prod2]) -> GrammarGen prod1 -> GrammarGen prod2 grammar_mapToProductions = fromMonadicLens grammar_mapToProductionsM @@ -36,14 +40,28 @@ data ProductionGen left right prod_right :: right } deriving (Eq, Ord, Show, Functor, Fold.Foldable, Traversable) +prod_mapToLeftM :: + Monad m => + (t -> m left) + -> ProductionGen t right -> m (ProductionGen left right) prod_mapToLeftM f p = do new <- f (prod_left p) return $ p{ prod_left = new } +prod_mapToRightM :: + Monad m => + (t -> m right) + -> ProductionGen left t -> m (ProductionGen left right) prod_mapToRightM f p = do new <- f (prod_right p) return $ p{ prod_right = new } +prod_mapToLeft :: + (a1 -> b) + -> ProductionGen a1 right -> ProductionGen b right prod_mapToLeft = fromMonadicLens prod_mapToLeftM +prod_mapToRight :: + (a1 -> b) + -> ProductionGen left a1 -> ProductionGen left b prod_mapToRight = fromMonadicLens prod_mapToRightM @@ -55,12 +73,16 @@ type Production = ProductionGen Var [Symbol] type Symbol = Either Terminal Var +isVar, isTerminal :: Symbol -> Bool isVar = Either.isRight isTerminal = Either.isLeft +epsilon :: Terminal epsilon = Terminal "" +varSymbol :: Var -> Symbol varSymbol = Right +terminalSymbol :: Terminal -> Symbol terminalSymbol = Left newtype Terminal = Terminal { @@ -79,15 +101,23 @@ data Tagged tag a = value :: a } deriving( Eq, Ord, Show) +tagged :: tag -> a -> Tagged tag a tagged = Tagged +tagged_mapToTagM :: + Monad m => + (a1 -> m tag) -> Tagged a1 a2 -> m (Tagged tag a2) tagged_mapToTagM f x = do new <- f $ tag x return $ x{ tag = new } +tagged_mapToValueM :: Monad m => + (a1 -> m a2) -> Tagged tag a1 -> m (Tagged tag a2) tagged_mapToValueM f x = do new <- f $ value x return $ x{ value = new } +tagged_mapToTag :: (a1 -> b) -> Tagged a1 a -> Tagged b a tagged_mapToTag = fromMonadicLens tagged_mapToTagM +tagged_mapToValue :: (a1 -> b) -> Tagged tag a1 -> Tagged tag b tagged_mapToValue = fromMonadicLens tagged_mapToValueM @@ -136,6 +166,7 @@ instance ToText Grammar where map pretty $ fromGrammar g + -- Pretty instance Pretty Var where pretty = var_name diff --git a/src/GrammarFormat.hs b/src/GrammarFormat.hs index f7e2801..fbf4b92 100644 --- a/src/GrammarFormat.hs +++ b/src/GrammarFormat.hs @@ -24,15 +24,25 @@ data GrammarFormat --grammarFormat_comment :: [(String, String)] } deriving (Show) + +type FormatMapToSurroundBy = (Maybe SurroundBy -> Maybe SurroundBy) -> GrammarFormat -> GrammarFormat +type FormatMapToStrings = ([String] -> [String]) -> GrammarFormat -> GrammarFormat + +gFormatMapToLeftSide :: FormatMapToSurroundBy gFormatMapToLeftSide f x = x{ grammarFormat_leftSide = f (grammarFormat_leftSide x) } +gFormatMapToRightSide :: FormatMapToSurroundBy gFormatMapToRightSide f x = x{ grammarFormat_rightSide = f (grammarFormat_rightSide x) } -gFormatMapToVar - :: (Maybe SurroundBy -> Maybe SurroundBy) - -> GrammarFormat -> GrammarFormat + +gFormatMapToVar :: FormatMapToSurroundBy gFormatMapToVar f x = x{ grammarFormat_var = f (grammarFormat_var x) } + +gFormatMapToTerminal :: FormatMapToSurroundBy gFormatMapToTerminal f x = x{ grammarFormat_terminal = f (grammarFormat_terminal x) } +gFormatMapToOr :: FormatMapToStrings gFormatMapToOr f x = x{ grammarFormat_or = f (grammarFormat_or x) } +gFormatMapToArrow :: FormatMapToStrings gFormatMapToArrow f x = x{ grammarFormat_arrow = f (grammarFormat_arrow x) } +gFormatMapToLineComment :: FormatMapToStrings gFormatMapToLineComment f x = x{ grammarFormat_lineComment = f (grammarFormat_lineComment x) } data AnnotationFormat @@ -58,6 +68,7 @@ data DefaultFormat | BNFE deriving (Show) +defaultFormat :: DefaultFormat -> GrammarFormat defaultFormat f = case f of Default -> def @@ -65,6 +76,7 @@ defaultFormat f = BNFE -> bnfe -- backus naur form +bnf :: GrammarFormat bnf = def{ grammarFormat_var = Just $ SurroundBy ("<",">"), @@ -74,6 +86,7 @@ bnf = } -- extended backus naur form +bnfe :: GrammarFormat bnfe = def{ grammarFormat_var = Nothing, @@ -81,6 +94,7 @@ bnfe = grammarFormat_arrow = ["::="] } +def :: GrammarFormat def = GrammarFormat { grammarFormat_leftSide = Nothing, diff --git a/src/GroupedGrammar.hs b/src/GroupedGrammar.hs index ee589be..da23d45 100644 --- a/src/GroupedGrammar.hs +++ b/src/GroupedGrammar.hs @@ -17,6 +17,7 @@ import GroupedGrammar.Conversions as Export import GroupedGrammar.Parse import Grammar.Types import GrammarFormat +import Parse.Format import Parse.ParseFormatFromGrammarFormat (parseFormatFromGrammarFormat) import Parse.Token import Types @@ -28,6 +29,7 @@ import qualified Data.Tree as Tree import qualified Data.Either as Either import qualified Data.Maybe as Maybe + toTextAsTree :: GrammarFormat -> GroupedGrammar_ProdAndSymbolsTagged ProductionTag [SymbolTag] -> String toTextAsTree format g = let @@ -49,11 +51,16 @@ toTextAsTree format g = :: Maybe (Tree.Tree (GroupedProduction_ProdAndSymbolsTagged ProductionTag [SymbolTag])) return $ fmap (toTextAs format) tree +groupedGrammarFromStr :: + ParseFormat -> String + -> Either String (GrammarGen (ProductionGen Var [[Either Terminal Var]])) groupedGrammarFromStr descr = groupedGrammarFromTokens <=< tokensFromStr descr +groupedGrammarFromTokens :: + [Token] -> Either String (GrammarGen (ProductionGen Var [[Either Terminal Var]])) groupedGrammarFromTokens = fmap (groupedGrammarNullProdsToEpsilonProds . groupedGrammarRebundle) . diff --git a/src/GroupedGrammar/Conversions.hs b/src/GroupedGrammar/Conversions.hs index 563a0fd..3359548 100644 --- a/src/GroupedGrammar/Conversions.hs +++ b/src/GroupedGrammar/Conversions.hs @@ -8,14 +8,6 @@ import Control.Monad import Control.Monad.Identity -{- -groupedGrammarNormalize :: GroupedGrammar -> GroupedGrammar -groupedGrammarNormalize = - grammar_mapToProductions $ - toGroupedProductions . - concatMap productionsFromGroupedProd --} - {- | this correctly groups productions which have equal left hand sides -} @@ -25,6 +17,9 @@ groupedGrammarRebundle = toGroupedProductions . concatMap productionsFromGroupedProd +groupedGrammarNullProdsToEpsilonProds :: + GrammarGen (ProductionGen left [[Either Terminal b]]) + -> GrammarGen (ProductionGen left [[Either Terminal b]]) groupedGrammarNullProdsToEpsilonProds = grammar_mapToProductions $ map $ prod_mapToRight $ @@ -37,6 +32,8 @@ groupedGrammarNullProdsToEpsilonProds = -- conversion: grouped <-> normal +grammarFromGroupedGrammar :: + GrammarGen GroupedProduction -> GrammarGen Production grammarFromGroupedGrammar ast = Grammar $ concatMap productionsFromGroupedProd $ @@ -45,6 +42,9 @@ grammarFromGroupedGrammar ast = productionsFromGroupedProd :: GroupedProduction -> [Production] productionsFromGroupedProd x = Production <$> [prod_left x] <*> prod_right x +asWithNormalProductions :: + ([Production] -> [Production]) + -> [GroupedProduction] -> [GroupedProduction] asWithNormalProductions f = runIdentity . diff --git a/src/GroupedGrammar/Parse.hs b/src/GroupedGrammar/Parse.hs index 9672f62..7ccdbf8 100644 --- a/src/GroupedGrammar/Parse.hs +++ b/src/GroupedGrammar/Parse.hs @@ -3,21 +3,23 @@ module GroupedGrammar.Parse where import Grammar.Types ---import GroupedGrammar.Internals +import GroupedGrammar.Types import Parse.Token import Types -import Text.Parsec hiding(many, (<|>)) +import Text.Parsec hiding(many, (<|>), tokens, token) import Control.Applicative -import Data.List +import Data.List( intercalate ) +parseGroupedGrammar :: Monad m => + ParsecT [Token] u m GroupedGrammar parseGroupedGrammar = fmap Grammar $ many $ do (VarToken varInfo) <- specificToken VarTokenType - specificToken ArrowTokenType + _ <- specificToken ArrowTokenType prod <- parseRightProdSide return $ Production (Var $ fromToken varInfo) prod @@ -44,6 +46,7 @@ parseRightProdSide = do TerminalToken info -> Left $ Terminal $ fromToken info _ -> error $ concat ["parseRightProdSide error: ", pretty t] +splitBy :: (a -> Bool) -> [a] -> [[a]] splitBy cond l = case l of [] -> [[]] @@ -67,11 +70,13 @@ parseSymbol = TerminalToken info -> Left $ Terminal $ fromToken info _ -> error "parseSymbol error" +oneOf' :: Monad m => [TokenType] -> ParsecT [Token] u m Token oneOf' tokT = tok ("token of type " ++ intercalate ", " (map show tokT) ++ " expected") ((`elem` tokT) . tokenType) +specificToken :: Monad m => TokenType -> ParsecT [Token] u m Token specificToken tokT = tok ("token of type " ++ show tokT ++ " expected") @@ -88,16 +93,3 @@ tok _ cond = let (line, col) = tokenPos token in (flip setSourceLine line . flip setSourceColumn col) pos - {- - updatePos pos token stream = - case tokenType token of - SepTokenType -> - incSourceLine pos 1 - _ -> - incSourceColumn pos 1 - -} -{- - do - t <- anyToken - if cond t then return t else parserFail errMsg --} diff --git a/src/GroupedGrammar/Transformations.hs b/src/GroupedGrammar/Transformations.hs index db74c36..f8e17fb 100644 --- a/src/GroupedGrammar/Transformations.hs +++ b/src/GroupedGrammar/Transformations.hs @@ -3,7 +3,6 @@ {-# LANGUAGE ScopedTypeVariables #-} module GroupedGrammar.Transformations( TransformationMonadT, TransformationMonad, - -- doLog, withNoLogging, Transformation(..), AnnotateInfo(..), VarCondition(..), InsertProductionsParams(..), @@ -36,9 +35,10 @@ import Utils.Graph import Types import qualified Data.Tree as Tree -import Data.List +import Data.List( find, (\\) ) import Data.Maybe import Control.Monad.Identity +import Control.Monad type TransformationMonadT m a = diff --git a/src/GroupedGrammar/Transformations/AddActionSymbols.hs b/src/GroupedGrammar/Transformations/AddActionSymbols.hs index 6c25692..119f1d4 100644 --- a/src/GroupedGrammar/Transformations/AddActionSymbols.hs +++ b/src/GroupedGrammar/Transformations/AddActionSymbols.hs @@ -1,12 +1,22 @@ module GroupedGrammar.Transformations.AddActionSymbols where ---import GroupedGrammar.Transformations.Types +import Grammar.Types +import GroupedGrammar.Types +import GroupedGrammar.Transformations.Types + import GroupedGrammar.Transformations.Utils import GroupedGrammar.Conversions -import Grammar.Types import Control.Monad.State + +addActionSymbols :: + Monad m => + Int + -> GroupedGrammarTagged [SymbolTag] + -> p1 + -> p2 + -> m (GroupedGrammar_SeparateProdTags prodTag [SymbolTag]) addActionSymbols counterInit grammar _ _ = return $ flip applyAlgorithmUsingProductions grammar $ diff --git a/src/GroupedGrammar/Transformations/AddProds.hs b/src/GroupedGrammar/Transformations/AddProds.hs index 801e06e..f7a4d0a 100644 --- a/src/GroupedGrammar/Transformations/AddProds.hs +++ b/src/GroupedGrammar/Transformations/AddProds.hs @@ -1,8 +1,17 @@ module GroupedGrammar.Transformations.AddProds where +import GroupedGrammar.Types + import GroupedGrammar.Transformations.Types import GroupedGrammar.Transformations.Utils + +insertProds :: Monad m => + InsertProductionsParams + -> GroupedGrammarTagged [SymbolTag] + -> p1 + -> p2 + -> m (GroupedGrammar_SeparateProdTags prodTag [SymbolTag]) insertProds params grammar _ _ = return $ flip applyAlgorithmUsingProductions grammar $ diff --git a/src/GroupedGrammar/Transformations/BreakProds.hs b/src/GroupedGrammar/Transformations/BreakProds.hs index 221860c..16f738e 100644 --- a/src/GroupedGrammar/Transformations/BreakProds.hs +++ b/src/GroupedGrammar/Transformations/BreakProds.hs @@ -2,15 +2,15 @@ {-# LANGUAGE FlexibleContexts #-} module GroupedGrammar.Transformations.BreakProds where +import Grammar.Types +import GroupedGrammar.Types + import GroupedGrammar.Transformations.Utils ---import GroupedGrammar.Types import GroupedGrammar.Conversions -import GroupedGrammar.Types -import Grammar.Types breakProds :: - (MonadLog m, MonadError String m) => + MonadLog m => VarScheme -> Int -> TransformationImplTypeM prodTag [SymbolTag] m diff --git a/src/GroupedGrammar/Transformations/DeleteProds.hs b/src/GroupedGrammar/Transformations/DeleteProds.hs index 15e6315..696d8a2 100644 --- a/src/GroupedGrammar/Transformations/DeleteProds.hs +++ b/src/GroupedGrammar/Transformations/DeleteProds.hs @@ -1,10 +1,19 @@ module GroupedGrammar.Transformations.DeleteProds where -import GroupedGrammar.Transformations.Utils import Grammar.Types +import GroupedGrammar.Types +import GroupedGrammar.Transformations.Types import Utils +import GroupedGrammar.Transformations.Utils + +deleteProds :: Monad m => + (Var -> Bool) + -> GroupedGrammarTagged [SymbolTag] + -> p1 + -> p2 + -> m (GroupedGrammar_SeparateProdTags prodTag [SymbolTag]) deleteProds varCond grammar _ _ = return $ let @@ -17,5 +26,6 @@ deleteProds varCond grammar _ _ = selectProd cond $ prods +deleteProd :: ([a], b, [a]) -> [a] deleteProd (preceding, _, remaining) = preceding ++ remaining diff --git a/src/GroupedGrammar/Transformations/ElimEpsilon.hs b/src/GroupedGrammar/Transformations/ElimEpsilon.hs index 1b9f09e..1f98a38 100644 --- a/src/GroupedGrammar/Transformations/ElimEpsilon.hs +++ b/src/GroupedGrammar/Transformations/ElimEpsilon.hs @@ -1,11 +1,19 @@ module GroupedGrammar.Transformations.ElimEpsilon where -import GroupedGrammar.Conversions -import GroupedGrammar.Transformations.Utils import Grammar.Types +import GroupedGrammar.Types +import GroupedGrammar.Transformations.Types + import Utils +import GroupedGrammar.Transformations.Utils +import GroupedGrammar.Conversions +elimEpsilon :: Monad m => + GroupedGrammarTagged [SymbolTag] + -> p1 + -> p2 + -> m (GroupedGrammar_SeparateProdTags prodTag [SymbolTag]) elimEpsilon grammar _ _ = let cond p = @@ -37,6 +45,9 @@ replaceAll prod prods = [] -> [Left epsilon] other -> other +appendIfChanged :: + Eq b => [Either Terminal b] -> [Either Terminal b] + -> [[Either Terminal b]] appendIfChanged old new = if new == old then [old] diff --git a/src/GroupedGrammar/Transformations/ElimLeftRecur.hs b/src/GroupedGrammar/Transformations/ElimLeftRecur.hs index fd48e9a..a640e30 100644 --- a/src/GroupedGrammar/Transformations/ElimLeftRecur.hs +++ b/src/GroupedGrammar/Transformations/ElimLeftRecur.hs @@ -3,28 +3,47 @@ module GroupedGrammar.Transformations.ElimLeftRecur where import GroupedGrammar.Transformations.VarNameMonad +import GroupedGrammar.Transformations.Types import GroupedGrammar.Conversions import GroupedGrammar.Types import GroupedGrammar.Transformations.Utils import Grammar.Types ---import Types import Utils -import Data.List ---import Data.Maybe ---import Control.Monad - ---import Debug.Trace +import Data.List( find, partition ) +elimLeftRecur_full :: + Monad m => + (Var -> Bool) + -> VarScheme + -> GrammarGen (ProductionGen Var [[Tagged [SymbolTag] Symbol]]) + -> p1 + -> p2 + -> m (GroupedGrammar_SeparateProdTags prodTag [SymbolTag]) elimLeftRecur_full = elimFull $ elimImmediateLeftRecursion +elimLeftRecurNoEpsilon_full :: Monad m => + (Var -> Bool) + -> VarScheme + -> GrammarGen (ProductionGen Var [[Tagged [SymbolTag] Symbol]]) + -> p1 + -> p2 + -> m (GroupedGrammar_SeparateProdTags prodTag [SymbolTag]) elimLeftRecurNoEpsilon_full = elimFull $ elimImmediateLeftRecursion_noEpsilon +elimFull :: Monad m => + (GroupedProduction -> VarNameMonadT m [GroupedProduction]) + -> (Var -> Bool) + -> VarScheme + -> GrammarGen (ProductionGen Var [[Tagged [SymbolTag] Symbol]]) + -> p1 + -> p2 + -> m (GroupedGrammar_SeparateProdTags prodTag [SymbolTag]) elimFull immediate varCond varScheme grammar _ _ = runVarNameMonadT varScheme @@ -37,26 +56,13 @@ elimFull immediate varCond varScheme grammar _ _ = . immediate ) $ elimIndirectLeftRecursion processed current -{- -elimLeftRecur_full = - elimFull $ - elimImmediateLeftRecursion - -elimLeftRecurNoEpsilon_full = - elimFull $ - elimImmediateLeftRecursion_noEpsilon - -elimFull immediate varCond varScheme = - applyAlgorithmUsingProductionsM varScheme $ - processAllOnceM $ - \(processed,remaining) current -> - ( - fmap (\p -> toGroupedProductions $ maybeUnfold varCond (processed++remaining) =<< productionsFromGroupedProd =<< p) - . immediate - ) $ - elimIndirectLeftRecursion processed current --} +elimLeftRecur :: Monad m => + VarScheme + -> GrammarGen (ProductionGen Var [[Tagged [SymbolTag] Symbol]]) + -> p1 + -> p2 + -> m (GroupedGrammar_SeparateProdTags prodTag [SymbolTag]) elimLeftRecur varScheme grammar _ _= runVarNameMonadT varScheme @@ -65,6 +71,12 @@ elimLeftRecur varScheme grammar _ _= processAllOnceM $ elimLeftRecurStep $ elimImmediateLeftRecursion +elimLeftRecurNoEpsilon :: Monad m => + VarScheme + -> GrammarGen (ProductionGen Var [[Tagged [SymbolTag] Symbol]]) + -> p1 + -> p2 + -> m (GroupedGrammar_SeparateProdTags prodTag [SymbolTag]) elimLeftRecurNoEpsilon varScheme grammar _ _= runVarNameMonadT varScheme @@ -74,7 +86,6 @@ elimLeftRecurNoEpsilon varScheme grammar _ _= elimLeftRecurStep $ elimImmediateLeftRecursion_noEpsilon elimLeftRecurStep :: - Monad m => (GroupedProduction -> VarNameMonadT m [GroupedProduction]) -> ProcessedAndRemaining GroupedProduction -> GroupedProduction -> VarNameMonadT m [GroupedProduction] @@ -134,12 +145,12 @@ elimImmediateLeftRecursionImpl splitProduction' prod = case partition isLeftRecursive rightSides of ([],_) -> return $ [prod] - partition -> -- (recursiveProductions, otherProductions) + partition' -> -- (recursiveProductions, otherProductions) fmap ( uncurry (splitProduction' left) $ mapFst (map $ drop 1) $ - partition + partition' ) $ getSimilarVar left where diff --git a/src/GroupedGrammar/Transformations/FindDeadEnds.hs b/src/GroupedGrammar/Transformations/FindDeadEnds.hs index 552c9b4..27c1f7c 100644 --- a/src/GroupedGrammar/Transformations/FindDeadEnds.hs +++ b/src/GroupedGrammar/Transformations/FindDeadEnds.hs @@ -2,20 +2,17 @@ module GroupedGrammar.Transformations.FindDeadEnds where import GroupedGrammar.Transformations.Utils ---import GroupedGrammar.Transformations.Types ---import GroupedGrammar.Conversions import GroupedGrammar.Types import Grammar.Types import qualified Utils.Graph as Graph ---import qualified Data.Map as M import qualified Data.Set as Set import qualified Data.Either as Either findDeadEnds :: - (MonadLog m, MonadError String m) => + (MonadLog m) => TransformationImplTypeM prodTag [SymbolTag] m findDeadEnds grammar _ graph = return $ @@ -25,20 +22,6 @@ findDeadEnds grammar _ graph = Set.toList $ allVars prods Set.\\ Set.fromList (Either.rights $ Graph.graph_nodeKeys graph) -{- -findDeadEnds :: - GroupedGrammarTagged [SymbolTag] - -> M.Map Var prodTag - -> Graph.Graph Symbol (GroupedProductionTagged [SymbolTag]) - -> Maybe (GroupedGrammar_SeparateProdTags prodTag [SymbolTag]) -findDeadEnds grammar m graph = - (\f -> applyAlgorithmUsingProductions f grammar m graph) $ - \prods -> - map (\var -> Production var []) $ - Set.toList $ - allVars prods Set.\\ Set.fromList (Either.rights $ Graph.graph_nodeKeys graph) --} - allVars :: [GroupedProduction] -> Set.Set Var allVars prods = foldr Set.union Set.empty $ diff --git a/src/GroupedGrammar/Transformations/FindLoops.hs b/src/GroupedGrammar/Transformations/FindLoops.hs index 0c6fa0d..0689495 100644 --- a/src/GroupedGrammar/Transformations/FindLoops.hs +++ b/src/GroupedGrammar/Transformations/FindLoops.hs @@ -46,9 +46,8 @@ findLoops startSymbol graph = do type SeedType = (Tree.Tree (GroupedProductionTagged [SymbolTag]), [GroupedProductionTagged [SymbolTag]]) findLoopsInTree :: GrammarGraph [SymbolTag] -> Tree.Tree (GroupedProductionTagged [SymbolTag]) -> Tree.Tree (GroupedProductionTagged [SymbolTag], [Var]) -findLoopsInTree graph tree = - Tree.unfoldTree unfoldF (tree, []) - :: Tree.Tree (GroupedProductionTagged [SymbolTag], [Var]) +findLoopsInTree graph tree_ = + Tree.unfoldTree unfoldF (tree_, []) where unfoldF :: SeedType -> ((GroupedProductionTagged [SymbolTag], [Var]), [SeedType]) unfoldF (tree, path) = (newNode, newSeeds) diff --git a/src/GroupedGrammar/Transformations/FirstSet.hs b/src/GroupedGrammar/Transformations/FirstSet.hs index 3e33031..b8eba9a 100644 --- a/src/GroupedGrammar/Transformations/FirstSet.hs +++ b/src/GroupedGrammar/Transformations/FirstSet.hs @@ -12,7 +12,8 @@ import qualified Data.Set as S import Data.Maybe ---annotateWithFirstSets :: +annotateWithFirstSets :: + GrammarGen GroupedProduction -> M.Map Var ProductionTag annotateWithFirstSets grammar = fmap (ProductionTag . Just) $ ( @@ -21,23 +22,6 @@ annotateWithFirstSets grammar = ) $ (initFirstSets $ fromGrammar $ grammar) -{- -annotateWithFirstSets :: - TransformationImplTypeM ProductionTag [SymbolTag] Maybe -annotateWithFirstSets ggTagged _ _ = - return $ - GroupedGrammar_SeparateProdTags { - ggSeparateProdTags_grammar = ggTagged, - ggSeparateProdTags_ruleAnnotations = - fmap (ProductionTag . Just) $ - ( - repeatTillNotChanging $ - step (concatMap productionsFromGroupedProd $ fromGrammar $ fromTaggedGrammar ggTagged) - ) $ - (initFirstSets $ fromGrammar $ fromTaggedGrammar ggTagged) - } --} - initFirstSets :: [GroupedProduction] -> M.Map Var FirstSet initFirstSets prods = M.fromList $ @@ -61,13 +45,13 @@ step prods = calcNewFirstSet (flip M.lookup firstMap) right calcNewFirstSet :: (Var -> Maybe FirstSet) -> [Symbol] -> FirstSet -calcNewFirstSet lookup right = +calcNewFirstSet lookupFunc right = case right of ((Left t):xs) -> ( if t == epsilon then - \x -> x `S.union` calcNewFirstSet lookup xs + \x -> x `S.union` calcNewFirstSet lookupFunc xs else id ) $ @@ -76,11 +60,11 @@ calcNewFirstSet lookup right = let directFirst = fromMaybe (S.singleton epsilon) $ - lookup var + lookupFunc var in if epsilon `S.member` directFirst then - directFirst `S.union` calcNewFirstSet lookup xs + directFirst `S.union` calcNewFirstSet lookupFunc xs else directFirst [] -> diff --git a/src/GroupedGrammar/Transformations/LeftFactor.hs b/src/GroupedGrammar/Transformations/LeftFactor.hs index 1c97fea..997fea8 100644 --- a/src/GroupedGrammar/Transformations/LeftFactor.hs +++ b/src/GroupedGrammar/Transformations/LeftFactor.hs @@ -16,17 +16,14 @@ import Control.Monad.State import qualified Data.Set as S import Data.Maybe ---import qualified Debug.Trace as Trace - leftFactor :: - (MonadLog m, MonadError String m) => + (MonadLog m) => VarScheme -> TransformationImplTypeM prodTag [SymbolTag] m leftFactor varScheme grammar _ _ = runVarNameMonadT varScheme (grammar_mapToProductions (map $ groupedProd_removeSymbolTags) $ grammar) $ flip applyAlgorithmUsingProductionsM grammar $ - --processAll_LeftFactoringM $ processAllOnceM $ \_ x -> fmap (uncurry (++)) $ @@ -55,20 +52,30 @@ data FullLeftFState = FullLeftFState { } type StopAtSet = S.Set GroupedProduction +fullLeftFState_mapToIt :: (Int -> Int) -> FullLeftFState -> FullLeftFState fullLeftFState_mapToIt f s = s{ fullLeftFState_it = f (fullLeftFState_it s) } + +fullLeftFState_mapToStopAt :: (StopAtSet -> StopAtSet) -> FullLeftFState -> FullLeftFState fullLeftFState_mapToStopAt f s = s{ fullLeftFState_stopAtTheseProds = f (fullLeftFState_stopAtTheseProds s) } +fullLeftFState_default :: FullLeftFState fullLeftFState_default = FullLeftFState 0 S.empty getIt :: MonadState FullLeftFState m => m Int getIt = fmap fullLeftFState_it $ get + incIt :: MonadState FullLeftFState m => m () incIt = modify $ fullLeftFState_mapToIt (+1) + getStopAt :: MonadState FullLeftFState m => m StopAtSet getStopAt = fmap fullLeftFState_stopAtTheseProds $ get + modifyStopAt :: MonadState FullLeftFState m =>(StopAtSet -> StopAtSet) -> m () modifyStopAt f = modify $ fullLeftFState_mapToStopAt f +unfoldCondFromDescr :: + FullLeftFactorIterateWhile -> Int + -> [GroupedProduction] -> GroupedProduction -> Bool unfoldCondFromDescr condDescr iteration unfolded prod = case condDescr of IterateWhileDecreasing -> @@ -110,7 +117,6 @@ leftFactor_full unfoldCondDescr varCond varScheme grammar _ _ = -> GroupedProduction -> VarNameMonadT m [GroupedProduction] transformProd unfoldCond _ prod = - --((lift $ doLog $ unlines $ ["leftFactorFull starting with", pretty $ prod_left prod]) >>) $ do (newProd, continuations) <- ((leftFactorProd prod) >>=) $ mapSndM $ @@ -127,7 +133,6 @@ leftFactor_full unfoldCondDescr varCond varScheme grammar _ _ = doLog $ "skipping unfolding of prod:" ++ pretty cont return $ [cont] unless (null continuations) $ - -- when (newProd/=prod) $ doLog $ unlines $ [ "splitting" @@ -150,20 +155,6 @@ type Path = [Node] instance MonadLog m => MonadLog (VarNameMonadT m) where doLog = lift . doLog -{- -leftFactorAndUnfold :: - Monad m => - (Var -> Bool) - -> [GroupedProduction] - -> GroupedProduction - -> VarNameMonadT m (GroupedProduction, [GroupedProduction]) -leftFactorAndUnfold cond allOtherProds input = - --((lift $ doLog $ unlines $ ["leftFactorAndUnfold"] ++ [pretty input]) >>) $ - fmap (mapSnd $ join . (map $ maybeUnfoldGrouped cond allOtherProds)) $ - leftFactorProd $ - input --} - maybeUnfoldGrouped :: (Var -> Bool) -> [GroupedProduction] @@ -218,29 +209,3 @@ leftFactorProd prod = ( Production left $ map fst input , catMaybes $ map snd $ input ) - -{- -joinProductions :: - [GroupedProduction] -> [GroupedProduction] -joinProductions = - map join_ . groupBy (\a b -> prod_left a == prod_left b) . sortOn prod_left - where - join_ productions = - case productions of - [] -> error "joinProductions error" - hd:_ -> Production (prod_left hd) $ concatMap prod_right productions - -logIfChanged :: MonadLog m => GroupedProduction -> [GroupedProduction] -> m [GroupedProduction] -logIfChanged old new = - do - when (length new /= 1) $ - doLog ( - concat $ - [ "expanded:\n\t" - , pretty old - , "\nto\n" - , unlines $ map (("\t" ++) . pretty) new - ] - ) - return $ new --} diff --git a/src/GroupedGrammar/Transformations/RemoveDoubleProds.hs b/src/GroupedGrammar/Transformations/RemoveDoubleProds.hs index 5f7b3a0..55b0cae 100644 --- a/src/GroupedGrammar/Transformations/RemoveDoubleProds.hs +++ b/src/GroupedGrammar/Transformations/RemoveDoubleProds.hs @@ -1,12 +1,18 @@ module GroupedGrammar.Transformations.RemoveDoubleProds where import GroupedGrammar.Transformations.Utils +import GroupedGrammar.Transformations.Types( GroupedGrammar_SeparateProdTags ) import GroupedGrammar.Types import Grammar.Types import Utils import Types +removeDoubleProds :: MonadLog m => + GroupedGrammarTagged [SymbolTag] + -> p1 + -> p2 + -> m (GroupedGrammar_SeparateProdTags prodTag [SymbolTag]) removeDoubleProds grammar _ _ = flip applyAlgorithmUsingProductionsM grammar $ repeatTillNotChangingM $ -- <- this is a hack, not very efficient... @@ -44,17 +50,6 @@ step input@(processed, remaining) prod = sym Nothing -> return $ (processed ++ [prod], remaining) - {- - | prod_right prod `elem` processed = - | otherwise = return $ processed ++ [prod] - -} - -{- -process f = - runIdentity - . - processM (return . f) --} processM :: Monad m => diff --git a/src/GroupedGrammar/Transformations/Types.hs b/src/GroupedGrammar/Transformations/Types.hs index 5e7a9c3..1b8f631 100644 --- a/src/GroupedGrammar/Transformations/Types.hs +++ b/src/GroupedGrammar/Transformations/Types.hs @@ -6,9 +6,6 @@ module GroupedGrammar.Transformations.Types where import GroupedGrammar.Types import Grammar.Types import Types ---import GrammarFormat ---import GrammarFormat ---import Types import Utils.Graph import Text.Regex.TDFA @@ -49,14 +46,6 @@ data VarScheme | FromVar -- originalName + number deriving Show -{- -data UnfoldParams = UnfoldParams { - unfoldParams_repeatUntilNotChanging :: Bool, - unfoldParams_varCondDescr :: VarCondition -} - deriving (Show) --} - data VarCondition = VarCondition { varCond_negate :: Bool, varCond_regex :: String @@ -82,37 +71,49 @@ data GrammarPosition varCondFromDescr :: VarCondition -> (Var -> Bool) varCondFromDescr descr = let - negate = varCond_negate descr + neg = varCond_negate descr regex = varCond_regex descr in - (if negate then not else id) + (if neg then not else id) . (=~regex) . var_name -{- -prodCondFromVarCondDescr :: VarCondition -> (ProductionGen Var right -> Bool) -prodCondFromVarCondDescr descr = - varCondFromDescr descr - . prod_left --} - data GroupedGrammar_SeparateProdTags productionTag symbolTag = GroupedGrammar_SeparateProdTags { ggSeparateProdTags_grammar :: GrammarGen (GroupedProductionTagged symbolTag), ggSeparateProdTags_ruleAnnotations :: M.Map Var productionTag } + + +ggSeparateProdTags_mapToGrammar :: + (GrammarGen (GroupedProductionTagged symbolTag1) -> GrammarGen (GroupedProductionTagged symbolTag2)) + -> GroupedGrammar_SeparateProdTags productionTag symbolTag1 + -> GroupedGrammar_SeparateProdTags productionTag symbolTag2 ggSeparateProdTags_mapToGrammar = fromMonadicLens ggSeparateProdTags_mapToGrammarM + +ggSeparateProdTags_mapToRuleAnnotations :: + (M.Map Var productionTag1 -> M.Map Var productionTag2) + -> GroupedGrammar_SeparateProdTags productionTag1 symbolTag + -> GroupedGrammar_SeparateProdTags productionTag2 symbolTag ggSeparateProdTags_mapToRuleAnnotations = fromMonadicLens ggSeparateProdTags_mapToRuleAnnotationsM +ggSeparateProdTags_mapToGrammarM :: Monad m => + (GrammarGen (GroupedProductionTagged symbolTag1) -> m (GrammarGen (GroupedProductionTagged symbolTag2))) + -> GroupedGrammar_SeparateProdTags productionTag symbolTag1 + -> m (GroupedGrammar_SeparateProdTags productionTag symbolTag2) ggSeparateProdTags_mapToGrammarM f g = do new <- f (ggSeparateProdTags_grammar g) return $ g{ ggSeparateProdTags_grammar = new } +ggSeparateProdTags_mapToRuleAnnotationsM :: Monad m => + (M.Map Var productionTag1 -> m (M.Map Var productionTag2)) + -> GroupedGrammar_SeparateProdTags productionTag1 symbolTag + -> m (GroupedGrammar_SeparateProdTags productionTag2 symbolTag) ggSeparateProdTags_mapToRuleAnnotationsM f g = do new <- f (ggSeparateProdTags_ruleAnnotations g) @@ -120,9 +121,6 @@ ggSeparateProdTags_mapToRuleAnnotationsM f g = type GrammarGraph symbolTag = Graph Symbol (GroupedProductionTagged symbolTag) - --Graph Symbol (GroupedProductionGen Var Symbol) ---type GrammarNode = Maybe Symbol --- Nothing means "or" graphFromGroupedGrammar :: Ord key => @@ -265,7 +263,7 @@ instance FromPretty VarScheme where case str of "%v%n" -> Right $ FromVar "" -> Left $ "fromPretty error reading VarScheme" - str -> Right $ Const str + _ -> Right $ Const str instance Pretty FullLeftFactorIterateWhile where pretty x = diff --git a/src/GroupedGrammar/Transformations/Unfold.hs b/src/GroupedGrammar/Transformations/Unfold.hs index d8ed801..01fde0b 100644 --- a/src/GroupedGrammar/Transformations/Unfold.hs +++ b/src/GroupedGrammar/Transformations/Unfold.hs @@ -1,14 +1,22 @@ module GroupedGrammar.Transformations.Unfold where import GroupedGrammar.Transformations.Utils +import GroupedGrammar.Transformations.Types( GroupedGrammar_SeparateProdTags ) import Grammar.Types import GroupedGrammar.Types import Utils -import Data.List +import Data.List( find ) import Data.Maybe +unfold :: + Monad m => + (Var -> Bool) + -> GroupedGrammarTagged [SymbolTag] + -> p1 + -> p2 + -> m (GroupedGrammar_SeparateProdTags prodTag [SymbolTag]) unfold varCond grammar _ _ = let cond = varCond . prod_left diff --git a/src/GroupedGrammar/Transformations/Utils.hs b/src/GroupedGrammar/Transformations/Utils.hs index a3c8a58..0c81662 100644 --- a/src/GroupedGrammar/Transformations/Utils.hs +++ b/src/GroupedGrammar/Transformations/Utils.hs @@ -30,6 +30,7 @@ type TransformationImplTypeM prodTag symbolTag m = type Zipper a = ([a], a, [a]) -- (preceding, selected, rest) +fromZipper :: ([a], a, [a]) -> [a] fromZipper (preceding, selected, rest) = preceding ++ [selected] ++ rest @@ -47,6 +48,10 @@ nextSelection cond (lastPreceding, lastSelected, lastRest) = \(preceding, selected, rest) -> (lastPreceding ++ [lastSelected] ++ preceding, selected, rest) +applyAlgorithmUsingProductions :: + ([ProductionGen Var [[Symbol]]] -> [ProductionGen Var [[Symbol]]]) + -> GroupedGrammarTagged [SymbolTag] + -> GroupedGrammar_SeparateProdTags prodTag [SymbolTag] applyAlgorithmUsingProductions f = runIdentity . applyAlgorithmUsingProductionsM (return . f) @@ -79,9 +84,11 @@ applyAlgorithmUsingProductionsM algorithm grammar = type ProcessedAndRemaining a = ([a],[a]) +untilM :: Monad m => (t -> Bool) -> (t -> m t) -> t -> m t untilM cond = untilExtM (\_ x -> cond x) +untilExt :: (c -> c -> Bool) -> (c -> c) -> c -> c untilExt cond f = runIdentity . untilExtM cond (return . f) @@ -97,6 +104,7 @@ untilExtM cond f x = then return $ new else untilExtM cond f new +untilExt_M :: (c -> c -> Bool) -> (c -> c) -> c -> c untilExt_M cond f = runIdentity . untilExtM cond (return . f) @@ -119,6 +127,9 @@ processAll f = runIdentity . processAllM (\processedAndRemaining current-> return $ f processedAndRemaining current) +processAllOnce :: + (ProcessedAndRemaining a -> Identity [a]) + -> [a] -> [a] processAllOnce f = runIdentity . processAllOnceM (return . f) @@ -132,20 +143,23 @@ processAllOnceM f = \processedAndRemaining currentElem -> fmap (,[]) $ f processedAndRemaining currentElem processAllM :: - forall a m . Monad m => + Monad m => (ProcessedAndRemaining a -> a -> m (ProcessedAndRemaining a)) -> [a] -> m [a] processAllM f l = fmap fst $ processAllM' f ([], l) - where - processAllM' :: (ProcessedAndRemaining a -> a -> m (ProcessedAndRemaining a)) -> ProcessedAndRemaining a -> m (ProcessedAndRemaining a) - processAllM' f partition@(processed, remaining) = - case remaining of - [] -> return $ partition - (x:xs) -> - do - (newElemsFinished, newElemsToBeProcessedAgain) <- f (processed,remaining) x - processAllM' f (processed ++ newElemsFinished, newElemsToBeProcessedAgain ++ xs) + +processAllM' :: + Monad m => + (ProcessedAndRemaining a -> a -> m (ProcessedAndRemaining a)) + -> ProcessedAndRemaining a -> m (ProcessedAndRemaining a) +processAllM' f partition@(processed, remaining) = + case remaining of + [] -> return $ partition + (x:xs) -> + do + (newElemsFinished, newElemsToBeProcessedAgain) <- f (processed,remaining) x + processAllM' f (processed ++ newElemsFinished, newElemsToBeProcessedAgain ++ xs) {- | `maybeUnfold cond allOtherProductions prod` does the following: diff --git a/src/GroupedGrammar/Transformations/VarNameMonad.hs b/src/GroupedGrammar/Transformations/VarNameMonad.hs index 572dba6..1c43080 100644 --- a/src/GroupedGrammar/Transformations/VarNameMonad.hs +++ b/src/GroupedGrammar/Transformations/VarNameMonad.hs @@ -17,6 +17,7 @@ import qualified Data.Either as Either import Data.Char import Control.Monad.State import Control.Monad.Identity +import Control.Monad type VarNameMonad a = VarNameMonadT Identity a @@ -31,43 +32,46 @@ data VarNameState = VarNameState { varNameState_varScheme :: VarScheme } +runVarNameMonad :: VarScheme -> GroupedGrammar -> VarNameMonadT Identity a -> a runVarNameMonad scheme g = runIdentity . runVarNameMonadT scheme g +varNameState_map :: (S.Set Var -> S.Set Var) -> VarNameState -> VarNameState varNameState_map f s = s{ fromVarNameState = f (fromVarNameState s) } getSimilarVar :: Monad m => Var -> VarNameMonadT m Var getSimilarVar var = VarNameMonad $ do - state <- get + st <- get let - varsSet = fromVarNameState state - scheme = varNameState_varScheme state + varsSet = fromVarNameState st + scheme = varNameState_varScheme st let newVar = calcNewVar scheme varsSet var modify $ varNameState_map $ (`S.union` S.singleton newVar) return $ newVar + +calcNewVar :: VarScheme -> S.Set Var -> Var -> Var +calcNewVar scheme varSet = + genFreeVar . getVar where - calcNewVar :: VarScheme -> S.Set Var -> Var -> Var - calcNewVar scheme varSet var' = - genFreeVar $ + getVar var = case scheme of - FromVar -> var' + FromVar -> var Const varName -> Var $ varName - where - genFreeVar var = - if var `S.member` varSet - then - let - varName = var_name var - (prefix, suffix) = spanEnd isDigit $ varName - in - genFreeVar $ - case suffix of - [] -> Var $ prefix ++ "0" - _ -> Var $ prefix ++ show ((read suffix :: Int) + 1) - else - var + genFreeVar var = + if var `S.member` varSet + then + let + varName = var_name var + (prefix, suffix) = spanEnd isDigit $ varName + in + genFreeVar $ + case suffix of + [] -> Var $ prefix ++ "0" + _ -> Var $ prefix ++ show ((read suffix :: Int) + 1) + else + var runVarNameMonadT :: Monad m => @@ -87,4 +91,5 @@ runVarNameMonadT scheme g m = \prod -> [prod_left prod] ++ join (map Either.rights (prod_right prod)) ) +spanEnd :: (a -> Bool) -> [a] -> ([a], [a]) spanEnd cond = (\(a,b) -> (reverse b, reverse a)) . span cond . reverse diff --git a/src/GroupedGrammar/Types.hs b/src/GroupedGrammar/Types.hs index 53ddb42..a487daf 100644 --- a/src/GroupedGrammar/Types.hs +++ b/src/GroupedGrammar/Types.hs @@ -7,14 +7,11 @@ module GroupedGrammar.Types where import Grammar.Types import Types import GrammarFormat ---import Utils (unlines) import Prelude hiding(unlines) -import Data.List hiding (unlines) +import Data.List( intercalate ) import qualified Data.Set as S ---import Data.Maybe - --------------------------------------------------- -- type synonyms for grouped grammars and productions @@ -45,7 +42,11 @@ data ProductionTag = ProductionTag { } deriving( Eq, Ord ) +prodTag_empty :: ProductionTag prodTag_empty = ProductionTag Nothing +prodTag_mapToFirstSet :: + (Maybe FirstSet -> Maybe FirstSet) + -> ProductionTag -> ProductionTag prodTag_mapToFirstSet f t = t{ prodTag_firstSet = f (prodTag_firstSet t) } type FirstSet = S.Set Terminal @@ -62,15 +63,27 @@ fromTaggedGrammar :: GroupedGrammarTagged symbolTag -> GroupedGrammar fromTaggedGrammar = fmap $ fromTaggedProduction +toTaggedProduction :: + ProductionGen left [[a1]] + -> ProductionGen left [[Tagged [a2] a1]] toTaggedProduction = prod_mapToRight $ map $ map $ Tagged [] +fromTaggedProduction :: + ProductionGen left [[Tagged tag b]] + -> ProductionGen left [[b]] fromTaggedProduction = prod_mapToRight $ map $ map $ value +groupedProd_removeSymbolTags :: + ProductionGen left [[Tagged tag b]] + -> ProductionGen left [[b]] groupedProd_removeSymbolTags = prod_mapToRight $ map $ map $ value - +groupedProd_addSymbolTags :: + tag + -> ProductionGen left [[a]] + -> ProductionGen left [[Tagged tag a]] groupedProd_addSymbolTags defTag = prod_mapToRight $ map $ map $ tagged defTag @@ -100,15 +113,17 @@ instance ToTextAs GrammarFormat ProductionTag where , intercalate ", " $ fmap (toTextAs format) $ S.toList $ set , "}" ] - {- - maybe "" id $ - (prodTag_firstSet x) >>= \set -> return $ - concat $ - [ "FIRST={" - , intercalate ", " $ fmap (toTextAs format) $ S.toList $ set - , "}" - ] - -} + +instance + ToTextAs GrammarFormat (GroupedProduction_ProdAndSymbolsTagged ProductionTag [SymbolTag]) where + toTextAs format p = + let + tagStr = toTextAs format $ tag p + prodStr = toTextAs format $ value p + in + if tagStr /= "" + then unwords [tagStr, prodStr] + else prodStr instance ( ToTextAs GrammarFormat left @@ -130,22 +145,19 @@ instance map (unwords . map (toTextTaggedSymbol format)) $ x +-- Pretty + instance - ToTextAs GrammarFormat (GroupedProduction_ProdAndSymbolsTagged ProductionTag [SymbolTag]) where - toTextAs format p = - let - tagStr = toTextAs format $ tag p - prodStr = toTextAs format $ value p - in - if tagStr /= "" - then unwords [tagStr, prodStr] - else prodStr - {- - unwords $ - [ toTextAs format $ tag p - , toTextAs format $ value p - ] - -} + (Pretty left, Pretty symbol) => + Pretty (ProductionGen left [[symbol]]) where + pretty p = + concat $ + [ pretty $ prod_left p + , " -> " + , intercalate " | " $ + map (unwords . map pretty) $ + prod_right p + ] toTextTaggedSymbol :: (ToTextAs GrammarFormat sym, ToTextAs GrammarFormat tag) => @@ -168,31 +180,3 @@ toTextTaggedSymbol format x = if annotationFormat_prepend annFormat then [tagStrSurrounded, valueStr] else [valueStr, tagStrSurrounded] - --- Pretty - -instance - (Pretty left, Pretty symbol) => - Pretty (ProductionGen left [[symbol]]) where - pretty p = - concat $ - [ pretty $ prod_left p - , " -> " - , intercalate " | " $ - map (unwords . map pretty) $ - prod_right p - ] - -{- -instance ToText GroupedGrammar where - toText groupedGrammar = - unlines $ - map (uncurry showEntry . (\x -> (prod_left x, prod_right x))) $ fromGroupedGrammar groupedGrammar - where - showEntry var rightRuleSides = - concat $ - [ pretty var - , " -> " - , intercalate " | " $ map (unwords . map pretty) rightRuleSides - ] --} diff --git a/src/Lib.hs b/src/Lib.hs new file mode 100644 index 0000000..5c244e5 --- /dev/null +++ b/src/Lib.hs @@ -0,0 +1,7 @@ +module Lib( +) where + +{- +someFunc :: IO () +someFunc = putStrLn "someFunc" +-} diff --git a/src/Parse/Format.hs b/src/Parse/Format.hs index 3b6fd89..a4bcedf 100644 --- a/src/Parse/Format.hs +++ b/src/Parse/Format.hs @@ -1,9 +1,6 @@ module Parse.Format where ---import GrammarFormat import Text.Parsec as P hiding(many, (<|>)) ---import Control.Monad ---import Control.Applicative data ParseFormat diff --git a/src/Parse/ParseFormatFromGrammarFormat.hs b/src/Parse/ParseFormatFromGrammarFormat.hs index 84e5132..25c8e70 100644 --- a/src/Parse/ParseFormatFromGrammarFormat.hs +++ b/src/Parse/ParseFormatFromGrammarFormat.hs @@ -10,6 +10,7 @@ import Control.Applicative import qualified Data.List as List +parseFormatFromGrammarFormat :: GrammarFormat -> ParseFormat parseFormatFromGrammarFormat grammarFormat = ParseFormat { parseFormat_symbol = @@ -30,6 +31,9 @@ parseFormatFromGrammarFormat grammarFormat = parseFormat_prodSep = map (P.try . P.string) $ grammarFormat_prodSep grammarFormat } +parseLineComment :: + Stream s m Char => + GrammarFormat -> ParsecT s u m String parseLineComment f = (choice $ map (P.try . P.string) $ grammarFormat_lineComment f) >> @@ -68,10 +72,16 @@ parseSymbol f stop = -- default: parse as terminal liftM Left $ parseUntilStop stop +parseSurround :: + Stream s m Char => + SurroundBy -> ParsecT s u m String parseSurround surround = let (prefix, suffix) = fromSurroundBy surround in P.string prefix *> (P.anyChar `P.manyTill` (P.lookAhead $ P.try $ P.string suffix)) <* P.string suffix +parseUntilStop :: + Stream s m Char => + ParsecT s u m end -> ParsecT s u m String parseUntilStop stop = P.anyChar `manyTill` (P.lookAhead stop) diff --git a/src/Parse/Token.hs b/src/Parse/Token.hs index bcba376..36e2dea 100644 --- a/src/Parse/Token.hs +++ b/src/Parse/Token.hs @@ -6,9 +6,13 @@ module Parse.Token( import Parse.Token.Internals import Parse.Token.Parse import Utils (mapLeft) +import Parse.Format( ParseFormat ) import qualified Text.Parsec as P +tokensFromStr :: + ParseFormat + -> String -> Either String [Token] tokensFromStr descr = mapLeft show . P.parse (parseTokens descr) "" diff --git a/src/Parse/Token/Internals.hs b/src/Parse/Token/Internals.hs index c560642..6b04e3a 100644 --- a/src/Parse/Token/Internals.hs +++ b/src/Parse/Token/Internals.hs @@ -1,3 +1,4 @@ +{-# OPTIONS_GHC -fno-warn-missing-signatures #-} module Parse.Token.Internals where diff --git a/src/Parse/Token/Parse.hs b/src/Parse/Token/Parse.hs index 1bec786..a73116d 100644 --- a/src/Parse/Token/Parse.hs +++ b/src/Parse/Token/Parse.hs @@ -1,5 +1,4 @@ {-# LANGUAGE ScopedTypeVariables #-} ---{-# LANGUAGE ExistentialQuantification #-} module Parse.Token.Parse where import Parse.Format @@ -9,13 +8,15 @@ import Utils (concLefts) import Text.Parsec hiding(many, (<|>)) import Control.Applicative import Data.Maybe +import Data.Functor.Identity + +parseTokens :: ParseFormat -> ParsecT String () Identity [Token] parseTokens descr = fmap ( deleteRepetitiveSeperators . mapMaybe (either (maybe Nothing Just) Just) - )$ - -- Either (Maybe Token) Token + ) $ ((try $ fmap Left $ sep descr) <|> fmap Right parseSymbol) `manyTill` (try $ lookAhead $ skipAtEndOfFile >> eof) @@ -37,6 +38,7 @@ parseTokens descr = ) >> return () +getPos :: ParsecT s u Identity (Text.Parsec.Line, Column) getPos = fmap (\p -> (sourceLine p, sourceColumn p)) getPosition @@ -47,6 +49,7 @@ deleteRepetitiveSeperators l = concLefts $ map (\t -> if tokenType t == SepTokenType then Left t else Right t) l +sep :: ParseFormat -> ParsecT String () Data.Functor.Identity.Identity (Maybe Token) sep descr = choice $ [ try $ parseComment descr *> return Nothing @@ -59,6 +62,9 @@ sep descr = , fmap (fmap SepToken) $ parseSepToken descr ] +type ParseFormatted res = ParseFormat -> ParsecT String () Identity res + +parseComment, parseWhitespace :: ParseFormatted [String] parseComment descr = many1 $ choice $ parseFormat_comment descr @@ -68,40 +74,23 @@ parseWhitespace descr = choice $ parseFormat_whitespaces descr +parseOrToken :: ParseFormatted (TokenInfo tokenType) parseOrToken descr = TokenInfo <$> (choice $ parseFormat_or descr) <*> getPos - -- >> return OrTokenInfo -parseSepToken :: - ParseFormat -> Parsec String () (Maybe SepTokenInfo) +parseSepToken :: ParseFormatted (Maybe SepTokenInfo) parseSepToken descr = (\seps pos -> return $ TokenInfo (concat seps) pos) <$> many1 ((choice $ parseFormat_prodSep descr) <* notFollowedBy (parseWhitespace descr)) <*> getPos - {- - ( - (choice $ parseFormat_prodSep descr) - `sepBy1` - (many $ ((try $ parseComment descr) <|> (parseWhitespace descr))) - ) - -} - {- - >> - return (Just SepTokenInfo) - -} - - {-((eof >> return Nothing) - <|> - return (Just SepTokenInfo)) - -} - --(eof >> return Nothing) <|> return (Just SepTokenInfo) +parseArrow :: ParseFormatted (TokenInfo tokenType) parseArrow descr = TokenInfo <$> @@ -109,8 +98,9 @@ parseArrow descr = parseFormat_prodSign descr) <*> getPos -parseOrSep p sep = - fmap Left (try sep) <|> fmap Right p +parseOrSep :: ParsecT s u m b -> ParsecT s u m a -> ParsecT s u m (Either a b) +parseOrSep p sepParser = + fmap Left (try sepParser) <|> fmap Right p parseStringOrSep :: Monad m => @@ -124,10 +114,3 @@ parseCharOrSep :: ParsecT String u m sep -> ParsecT String u m (Either Char sep) parseCharOrSep parseSep = fmap Right (try parseSep) <|> fmap Left anyChar - -{- -testTokenParser :: - String -> Either ParseError [String] -testTokenParser = - parse (parseToken descr <* eof) "" --} diff --git a/src/Types.hs b/src/Types.hs index bd9f10d..d904875 100644 --- a/src/Types.hs +++ b/src/Types.hs @@ -1,6 +1,7 @@ {-# LANGUAGE MultiParamTypeClasses #-} {-# LANGUAGE TypeSynonymInstances #-} {-# LANGUAGE FlexibleInstances #-} +{-# LANGUAGE Rank2Types #-} module Types where import Control.Monad.Identity @@ -32,5 +33,8 @@ instance ToText String where toText = id -- helper to implement lenses by hand: +fromMonadicLens :: + (forall m . Monad m => (a1 -> m b) -> a2 -> m c) + -> (a1 -> b) -> a2 -> c fromMonadicLens lens f = runIdentity . lens (return . f) diff --git a/src/Utils.hs b/src/Utils.hs index 74ac930..c6d554b 100644 --- a/src/Utils.hs +++ b/src/Utils.hs @@ -6,28 +6,34 @@ module Utils( module Control.Monad.Except ) where -import Data.List as List import Control.Monad.Except import Control.Monad +import Data.List( sortBy ) ---unlines = List.intercalate "\n" - +mapFst :: (a -> t) -> (a, b) -> (t, b) mapFst f (a, b) = (f a, b) +mapSnd :: (b -> t) -> (a, b) -> (a, t) mapSnd f (a, b) = (a, f b) +mapFstM :: Monad m => (a -> m t) -> (a, b) -> m (t, b) mapFstM f (a, b) = fmap (, b) $ f a +mapSndM :: Monad m => (b -> m t) -> (a, b) -> m (a, t) mapSndM f (a, b) = fmap (a, ) $ f b +isLeft :: Either a b -> Bool isLeft (Right _) = False isLeft (Left _) = True +mapLeft :: (l -> t) -> Either l r -> Either t r mapLeft f (Left x) = Left (f x) mapLeft _ (Right x) = Right x +mapRight :: (r -> t) -> Either l r -> Either l t mapRight _ (Left x) = Left x mapRight f (Right x) = Right (f x) +uncurry3 :: (t1 -> t2 -> t3 -> t4) -> (t1, t2, t3) -> t4 uncurry3 f (a,b,c) = f a b c concLefts :: @@ -47,36 +53,19 @@ concLefts l = [] -> [] (x:xs) -> mapLeft (const []) x : concLefts xs -{- -{- - if string a is a prefix of string b, then b will come before a - ex.: - orderByPrefix ["a", "ab", "cde"] == ["ab", "a", "cde"] --} -orderByPrefix :: Ord a => [[a]] -> [[a]] -orderByPrefix = sortBy cmp - where - cmp [] [] = EQ - cmp [] (_:_) = GT - cmp (_:_) [] = LT - cmp (x:xs) (y:ys) = - case compare x y of - EQ -> cmp xs ys - other -> other --} -- if two or more lists have the same prefix (/= epsilon), they are returned as a group. {- e.g. groupByPrefix ["abcdef", "abc", "abcHURZ", "xyz"] == [("xyz",[]), ("abc",["HURZ", "", "def"]) ] -} groupByPrefix :: - (Eq a, Ord a) => [[a]] -> [([a], [[a]])] + Ord a => [[a]] -> [([a], [[a]])] groupByPrefix l = reverse $ -- <- quick and dirty solution - foldl conc init $ + foldl conc start $ sortBy lexic l where - conc :: (Eq a, Ord a) => [([a],[[a]])] -> [a] -> [([a],[[a]])] + conc :: (Eq a) => [([a],[[a]])] -> [a] -> [([a],[[a]])] conc res l2 = case res of ((pref, rests):xs) -> @@ -90,9 +79,10 @@ groupByPrefix l = _ -> (l2,[[]]):res _ -> [(l2, [[]])] - init = [] + start = [] -- |note: if a is a prefix of b, then a < b +lexic :: Ord a => [a] -> [a] -> Ordering lexic a b = case (a,b) of ([],[]) -> EQ diff --git a/src/Utils/Graph.hs b/src/Utils/Graph.hs index 580aa19..4834359 100644 --- a/src/Utils/Graph.hs +++ b/src/Utils/Graph.hs @@ -21,6 +21,7 @@ graph_nodeKeys g = where xtract (_, x, _) = x +graphFromEdges :: Ord key => [(node, key, [key])] -> Graph key node graphFromEdges = uncurry3 Graph . G.graphFromEdges @@ -33,12 +34,14 @@ spanningForest vars graph = ) $ mapM (graph_vertexFromKey graph) $ vars +lookupVertex :: Graph key node -> Int -> (key, node) lookupVertex graph vertex = let (node, key, _) = graph_nodeFromVertex graph vertex in (key, node) -- all nodes reachable from a given node +hull :: Graph key node -> key -> Maybe [(key, node)] hull graph key = fmap ( map (lookupVertex graph) . @@ -46,27 +49,8 @@ hull graph key = ) $ graph_vertexFromKey graph key +isReachable :: Graph key node -> key -> key -> Maybe Bool isReachable graph k1 k2 = G.path (graph_graph graph) <$> (graph_vertexFromKey graph $ k1) <*> (graph_vertexFromKey graph $ k2) - -{- ---spanningForest :: [Var] -> GrammarGraph -> Maybe [Tree.Tree GroupedProduction] -spanningForest vars graph = - do - keyVertices <- - mapM (graph_vertexFromKey graph . Right) $ vars - :: Maybe [G.Vertex] - let spanningForest = G.dfs (graph_graph graph) keyVertices :: [Tree.Tree G.Vertex] - return $ - map (fmap $ lookupVertex) $ - spanningForest - where - lookupVertex vertex = - let (node, key, destinations) = graph_nodeFromVertex graph vertex - in - case key of - (Right var) -> node - _ -> error "spanning forest error" --} diff --git a/src/Utils/Logging.hs b/src/Utils/Logging.hs index 13eaafd..8d09e46 100644 --- a/src/Utils/Logging.hs +++ b/src/Utils/Logging.hs @@ -23,9 +23,3 @@ instance MonadLog IO where instance MonadLog m => MonadLog (ExceptT e m) where doLog = lift . doLog - - -{- -instance (Monad (t m), MonadLog m, MonadTrans t) => MonadLog (t m) where - doLog = lift . doLog --} diff --git a/stack.yaml b/stack.yaml new file mode 100644 index 0000000..047d0d5 --- /dev/null +++ b/stack.yaml @@ -0,0 +1,67 @@ +# This file was automatically generated by 'stack init' +# +# Some commonly used options have been documented as comments in this file. +# For advanced use and comprehensive documentation of the format, please see: +# https://docs.haskellstack.org/en/stable/yaml_configuration/ + +# Resolver to choose a 'specific' stackage snapshot or a compiler version. +# A snapshot resolver dictates the compiler version and the set of packages +# to be used for project dependencies. For example: +# +# resolver: lts-3.5 +# resolver: nightly-2015-09-21 +# resolver: ghc-7.10.2 +# +# The location of a snapshot can be provided as a file or url. Stack assumes +# a snapshot provided as a file might change, whereas a url resource does not. +# +# resolver: ./custom-snapshot.yaml +# resolver: https://example.com/snapshots/2018-01-01.yaml +resolver: + url: https://raw.githubusercontent.com/commercialhaskell/stackage-snapshots/master/lts/22/11.yaml + +# User packages to be built. +# Various formats can be used as shown in the example below. +# +# packages: +# - some-directory +# - https://example.com/foo/bar/baz-0.0.2.tar.gz +# subdirs: +# - auto-update +# - wai +packages: +- . +# Dependency packages to be pulled from upstream that are not in the resolver. +# These entries can reference officially published versions as well as +# forks / in-progress versions pinned to a git hash. For example: +# +# extra-deps: +# - acme-missiles-0.3 +# - git: https://github.com/commercialhaskell/stack.git +# commit: e7b331f14bcffb8367cd58fbfc8b40ec7642100a +# +# extra-deps: [] + +# Override default flag values for local packages and extra-deps +# flags: {} + +# Extra package databases containing global packages +# extra-package-dbs: [] + +# Control whether we use the GHC we find on the path +# system-ghc: true +# +# Require a specific version of stack, using version ranges +# require-stack-version: -any # Default +# require-stack-version: ">=2.7" +# +# Override the architecture used by stack, especially useful on Windows +# arch: i386 +# arch: x86_64 +# +# Extra directories used by stack for building +# extra-include-dirs: [/path/to/dir] +# extra-lib-dirs: [/path/to/dir] +# +# Allow a newer minor version of GHC than the snapshot specifies +# compiler-check: newer-minor diff --git a/stack.yaml.lock b/stack.yaml.lock new file mode 100644 index 0000000..433458b --- /dev/null +++ b/stack.yaml.lock @@ -0,0 +1,13 @@ +# This file was autogenerated by Stack. +# You should not edit this file by hand. +# For more information, please see the documentation at: +# https://docs.haskellstack.org/en/stable/lock_files + +packages: [] +snapshots: +- completed: + sha256: 2fdd7d3e54540062ef75ca0a73ca3a804c527dbf8a4cadafabf340e66ac4af40 + size: 712469 + url: https://raw.githubusercontent.com/commercialhaskell/stackage-snapshots/master/lts/22/11.yaml + original: + url: https://raw.githubusercontent.com/commercialhaskell/stackage-snapshots/master/lts/22/11.yaml diff --git a/test/Spec.hs b/test/Spec.hs new file mode 100644 index 0000000..cd4753f --- /dev/null +++ b/test/Spec.hs @@ -0,0 +1,2 @@ +main :: IO () +main = putStrLn "Test suite not yet implemented"