C# Parser

CREATED ISSUE: Enums are handled like casts

RobUrsem — Fri, 06 Jun 2008 21:33:16 GMT

Case statements where enums are used in the form of MyEnum.A will crash the parser with an exception:

System.InvalidOperationException was unhandled
Message="Stack empty."
Source="System"
StackTrace:
at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
at System.Collections.Generic.Stack`1.Pop()
at DDW.Parser.ParseCastOrGroup() in D:\Projects\CSParser\CSLexer\Parser.cs:line 5591
at DDW.Parser.ParseExpressionSegment() in D:\Projects\CSParser\CSLexer\Parser.cs:line 5314
at DDW.Parser.ParseExpression() in D:\Projects\CSParser\CSLexer\Parser.cs:line 4806
at DDW.Parser.ParseCase() in D:\Projects\CSParser\CSLexer\Parser.cs:line 4248
at DDW.Parser.ParseSwitch() in D:\Projects\CSParser\CSLexer\Parser.cs:line 4231
at DDW.Parser.ParseStatement(NodeCollection`1 node) in D:\Projects\CSParser\CSLexer\Parser.cs:line 3943
at DDW.Parser.ParseBlock(BlockStatement node, Boolean isCase) in D:\Projects\CSParser\CSLexer\Parser.cs:line 4147
at DDW.Parser.ParseBlock(BlockStatement node) in D:\Projects\CSParser\CSLexer\Parser.cs:line 4128
at DDW.Parser.ParseMethod(IType type, QualifiedIdentifierExpression name) in D:\Projects\CSParser\CSLexer\Parser.cs:line 2576
at DDW.Parser.ParseClassMember() in D:\Projects\CSParser\CSLexer\Parser.cs:line 1532
at DDW.Parser.ParseClass() in D:\Projects\CSParser\CSLexer\Parser.cs:line 1031
at DDW.Parser.ParseNamespaceOrTypes() in D:\Projects\CSParser\CSLexer\Parser.cs:line 287
at DDW.Parser.ParseNamespace() in D:\Projects\CSParser\CSLexer\Parser.cs:line 756
at DDW.Parser.ParseNamespaceOrTypes() in D:\Projects\CSParser\CSLexer\Parser.cs:line 283
at DDW.Parser.Parse(TokenCollection tokens, List`1 strings) in D:\Projects\CSParser\CSLexer\Parser.cs:line 213
at DDW.Cmc.ParseFile(String fileName, List`1 errors) in D:\Projects\CSParser\Test program\cmc.cs:line 113
at DDW.Cmc.Main(String[] args) in D:\Projects\CSParser\Test program\cmc.cs:line 80
at System.AppDomain._nExecuteAssembly(Assembly assembly, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()

CREATED ISSUE: newest version does not like #region

ChrisParkinson — Mon, 24 Mar 2008 16:38:31 GMT

using the newest version from the source code tab it does not seem to like regions. Try parsing the Parser.cs.

problem seems to be in the Lex method of Lexer.cs in the identifiers and keywords region where the identifier is determined to be "region Hello" rather than "region" in my test case.

I think the problem is in here
while (c != -1 && !endIdent) {
convertedChar = (char)c;

if (!isHashLine && !Char.IsLetterOrDigit(convertedChar) && convertedChar != '_' ||
(c == '\n' || c == '\r')) {
endIdent = true;
}
else {
//do not use code like "if c =='a'" because it might be not latin caharacter
if (!Char.IsLetter(convertedChar) || Char.IsUpper(convertedChar)) {
possibleKeyword = false;
}

sb.Append(convertedChar);
c = src.Read();
curCol++;
}
}
any ideas where to fix this?

thanks

Chris

COMMENTED TASK: Create a generator

AdamStevenson — Thu, 20 Mar 2008 05:09:20 GMT

My personal goal is to use this to target microprocessors (like the SX chip, or more generally Pics). In any case it would be nice to attach some sort of generator to this - maybe there is an IL writer out there somewhere that needs a friend : ).
Comments: ** Comment from web user: AdamStevenson **

I would be interested in working with someone on this. :)

NEW POST: C# CodeDOM

AdamStevenson — Thu, 20 Mar 2008 05:05:03 GMT

I would also be interested in getting a backend written for the compiler as well.

NEW POST: C# CodeDOM

AdamStevenson — Thu, 20 Mar 2008 04:51:28 GMT

Hey, just fyi, I am going to be needing to work on a unified reflection / codedom model for C# for my thesis project. IF anyone is interested, shoot me an email or post a reply here.

CREATED ISSUE: Lexer crashes when trying to tokenize '\0'

davec — Tue, 18 Mar 2008 05:03:33 GMT

When attempting to tokenize the NUL character escape sequence the lexer crashes. This is because the case block for the NUL checks the second character against the integer 0 rather than the character '0'. Simple patch to fix this is attached.

NEW POST: Documentation?

formalist — Fri, 22 Feb 2008 14:22:44 GMT

Hello,

sorry for the silly question, but... is there any documentation? (one 'HowTo' would be very appreciated)

Thank you.

Patch Uploaded

mkroll — Mon, 07 Jan 2008 18:30:56 GMT

mkroll has uploaded a patch.

Description:
Fixes switch cases by replacing BlockStatement by a list of Statements in CaseNode.
This resolves workitems 11771, 11772, and 11773.
(It also removes the space after true and false)

With this patch you can finally parse the parser with itself, unparse it, compile the result, parse it again, unparse, compile, ...

COMMENTED ISSUE: Doesn't handle properties properly

mkroll — Mon, 07 Jan 2008 17:52:33 GMT

In the attached file,
public int Test2
{
get { return 0; }
set { _i = value; }
}

After parsing, only the Getter property is set, with Kind == "set".

Line 1126 (in release 0.5) should read:

if (curtok.ID == TokenID.Ident && strings[curtok.Data] == "set")
{
node.Setter = ParseAccessor();
}

(at the moment, it's node.Getter = ParseAccessor)
Comments: ** Comment from web user: mkroll **

Is fixed in the current revision (29232) from 2007-01-19.

COMMENTED ISSUE: Parser.cs deadlock

mkroll — Mon, 07 Jan 2008 16:47:39 GMT

Hello, I'm trying to parse a specific file, which I'm attaching, but the parser seems to hang at line 1029 of Parser.cs file, entering an infinite loop.
Comments: ** Comment from web user: mkroll **

Btw, the infinite loop is fixed by Patch 605.

COMMENTED ISSUE: Parser.cs deadlock

mkroll — Mon, 07 Jan 2008 16:36:54 GMT

Hello, I'm trying to parse a specific file, which I'm attaching, but the parser seems to hang at line 1029 of Parser.cs file, entering an infinite loop.
Comments: ** Comment from web user: mkroll **

Patch 604 solves this problem as the preprocessor was broken and a wrong token stream was generated.

Patch Uploaded

mkroll — Mon, 07 Jan 2008 12:28:35 GMT

mkroll has uploaded a patch.

Description:
Fixes unparsing abstract methods.

Patch Uploaded

mkroll — Mon, 07 Jan 2008 12:09:36 GMT

mkroll has uploaded a patch.

Description:
Fixes unparsing of assembly attributes.

NEW POST: C# CodeDOM

nesteruk — Fri, 04 Jan 2008 15:22:13 GMT

Well, if you guys aren't doing CodeDOM, maybe you can create something of anequivalent. I mean, one of the goals of these parsers is to make facilities for analysing and querying code, which is difficult without an object model.

NEW POST: C# CodeDOM

ovanklot — Sat, 29 Dec 2007 11:12:13 GMT

Hi Yansong,

I'm sorry, but I've pretty much abandoned this project in the past few months due to lack of time and well... interest on my side. You see - since Microsoft have simply decided to kill off CodeDom, which is my expression for not updating, debugging or even talking about it in the past two versions of the Framework, I don't see a point in this continuing as a project.
Sorry.

Omer

NEW POST: C# CodeDOM

Yansong — Fri, 28 Dec 2007 20:21:20 GMT

Hi, huw and Omer

I am interested in knowing what the progress is for th ToCodeDom project. I am looking for a C# parser that is fully compatible with Microsoft's CodeDom. I would like to see that a code compile unit object be returned from the parser so that i can generate or compile code on the fly. I downloaded the latest source code which supports C# 2.0. But I did not seem to find anything refering to Code Compile Unit. Please kindly keep me in the loop by posting here or email me directly.

Thanks,
Yansong

Patch Uploaded

mkroll — Thu, 20 Dec 2007 18:59:20 GMT

mkroll has uploaded a patch.

Description:
Fixes "foreach" parsing and unparsing.

COMMENTED ISSUE: Opposite order of local declarations

mkroll — Thu, 20 Dec 2007 16:23:31 GMT

The code

class testx2
{
void test()
{
int x = 0, y = x;
}
}

is translated to

class testx2
{
void test()
{
int y = x;
int x = 0;

}

}

because Parser.PopLocalDeclaration() adds the LocalDeclarationStatements in the opposite order of occurrence.
I would suggest supplying LocalDeclarationStatement with a list of identifier/expression pairs instead of splitting a declaration into multiple statements to keep the result bound to the input.
Comments: ** Comment from web user: mkroll **

I have posted a patch implementing the above suggestion.
As mentioned there, the patch also fixes an infinite loop when a syntax error occurs in a local declaration.

Patch Uploaded

mkroll — Thu, 20 Dec 2007 16:20:53 GMT

mkroll has uploaded a patch.

Description:
This patch fixes local declarations (see work item #14491) and adds support for multiple alternatively expected tokens to RecoverFromError.
This also fixes an infinite loop when a syntax error occurs in a local declaration.

Patch Uploaded

mkroll — Thu, 20 Dec 2007 14:45:22 GMT

mkroll has uploaded a patch.

Description:
This patch fixes preprocessor directives. The lexer did not parse idents/keywords in preprocessor directives correctly.