Skip to content

Commit

Permalink
Merge pull request universal-ctags#927 from masatake/automatic-full-q…
Browse files Browse the repository at this point in the history 
…ualified-tag-generation-in-cork

Automatic full qualified tag generation in cork
  • Loading branch information
@masatake
masatake committed May 19, 2016
2 parents 55aa7ad + 70a63d1 commit 3c9a4fe
Show file tree
Hide file tree
Showing 21 changed files with 171 additions and 42 deletions.
3 changes: 2 additions & 1 deletion Tmain/list-fields-with-prefix.d/stdout-expected.txt
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
r UCTAGSrole off NONE TRUE Role
Z UCTAGSscope off NONE FALSE Include the "scope:" key in scope field (use s)
Z UCTAGSscope off NONE TRUE Include the "scope:" key in scope field (use s) in tags output, scope name in xref output
E UCTAGSextra off NONE TRUE Extra tag type information
x UCTAGSxpath off NONE TRUE xpath for the tag
p UCTAGSscopeKind on NONE TRUE Kind of scope as full name
- UCTAGSend off C TRUE end lines of various constructs
- UCTAGSproperties off C TRUE properties (static, inline, mutable,...)
- UCTAGSend off C++ TRUE end lines of various constructs
Expand Down
5 changes: 3 additions & 2 deletions Tmain/list-fields.d/stdout-expected.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,15 @@ l language off NONE TRUE Language of input file containing tag
m implementation off NONE TRUE Implementation information
n line off NONE TRUE Line number of tag definition
S signature off NONE TRUE Signature of routine (e.g. prototype or parameter list)
s NONE on NONE TRUE Scope of tag definition (WARNING: this doesn't work well as a format letter)
s NONE on NONE TRUE Scope of tag definition (`p' can be used for printing its kind)
t typeref on NONE TRUE Type and name of a variable or typedef
z kind off NONE FALSE Include the "kind:" key in kind field (use k or K)
r role off NONE TRUE Role
R NONE off NONE TRUE Marker (R or D) representing whether tag is definition or reference
Z scope off NONE FALSE Include the "scope:" key in scope field (use s)
Z scope off NONE TRUE Include the "scope:" key in scope field (use s) in tags output, scope name in xref output
E extra off NONE TRUE Extra tag type information
x xpath off NONE TRUE xpath for the tag
p scopeKind on NONE TRUE Kind of scope as full name
- end off C TRUE end lines of various constructs
- properties off C TRUE properties (static, inline, mutable,...)
- end off C++ TRUE end lines of various constructs
Expand Down
0 Units/dts-core-dump.d/input.dts → Units/parser-dts.r/dts-core-dump.d/input.dts
View file
File renamed without changes.
6 changes: 6 additions & 0 deletions Units/parser-dts.r/dts-fq-without-scope-field.d/README
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
This test case verifies a corner use case of Cork based automatic full
qualified tag generator(FQgen).

FQgen caches scope informations built just before emitting it as value
for scope fields. Therefore when --field=-s(disabling scope field) is
given, FQ tags were not generated.
2 changes: 2 additions & 0 deletions Units/parser-dts.r/dts-fq-without-scope-field.d/args.ctags
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
--fields=+Zr-s
--extra=+rq
8 changes: 8 additions & 0 deletions Units/parser-dts.r/dts-fq-without-scope-field.d/expected.tags
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
0x00 input.dts /^ phandle = <0x00>;$/;" p
bar input.dts /^#define bar /;" d file:
bar input.dts /^#undef bar$/;" d file: role:undef
foo.dtsi input.dts /^#include "foo.dtsi"/;" h role:local
label input.dts /^label: test {$/;" l
label.0x00 input.dts /^ phandle = <0x00>;$/;" p
label.label2 input.dts /^ label2: chosen { } ;$/;" l
label2 input.dts /^ label2: chosen { } ;$/;" l
0 Units/dts-simple.d/input.dts → ....r/dts-fq-without-scope-field.d/input.dts
View file
File renamed without changes.
2 changes: 1 addition & 1 deletion Units/dts-simple.d/args.ctags → Units/parser-dts.r/dts-simple.d/args.ctags
View file
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
--fields=+Zr
--extra=+r
--extra=+rq
2 changes: 2 additions & 0 deletions Units/dts-simple.d/expected.tags → ...s/parser-dts.r/dts-simple.d/expected.tags
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,6 @@ bar input.dts /^#define bar /;" d file:
bar input.dts /^#undef bar$/;" d file: role:undef
foo.dtsi input.dts /^#include "foo.dtsi"/;" h role:local
label input.dts /^label: test {$/;" l
label.0x00 input.dts /^ phandle = <0x00>;$/;" p scope:label:label
label.label2 input.dts /^ label2: chosen { } ;$/;" l scope:label:label
label2 input.dts /^ label2: chosen { } ;$/;" l scope:label:label
12 changes: 12 additions & 0 deletions Units/parser-dts.r/dts-simple.d/input.dts
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
/dts-v1/;

#include "foo.dtsi"
#define bar 98

label: test {
dummy {
label2: chosen { } ;
phandle = <0x00>;
};
};
#undef bar
29 changes: 27 additions & 2 deletions docs/internal.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ parser does not set `useCork` field. `writeTagEntry` calls one of
three functions, `writeTagsEntry`, `writeXrefEntry` or `writeCtagsEntry`.
One of them is chosen depending on the arguments passed to ctags.

If `useCork` is set, the tag informations goes to a queue on memory.
If `useCork` is set, the tag information goes to a queue on memory.
The queue is flushed when `useCork` in unset. See `cork API` for more
details.

Expand All @@ -121,7 +121,7 @@ Following code is taken from clojure.c(with modifications).
makeTagEntry (&current);
`parent`, values stored to `scope [0]` and `scope [1]` are all
something string.
kind of strings.

cork API provides more solid way to hold scope information. cork API
expects `parent`, which represents scope of a tag(`current`)
Expand Down Expand Up @@ -199,3 +199,28 @@ When using `scopeIndex` of `current`, `NULL` must be assigned to both
`current.extensionFields.scope[1]`. `initTagEntry` function does this
initialization internally, so you generally you don't have to write
the initialization explicitly.

Automatic full qualified tag generation
,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

If a parser uses the cork for recording and emitting scope
information, ctags can reuse it for generating full qualified(FQ)
tags. Set `requestAutomaticFQTag` field of `parserDefinition` to
`TRUE` then the main part of ctags emits FQ tags on behalf of the parser
if `--extra=+q` is given.

An example can be found in DTS parser:

.. code-block:: c
extern parserDefinition* DTSParser (void)
{
static const char *const extensions [] = { "dts", "dtsi", NULL };
parserDefinition* const def = parserNew ("DTS");
...
def->requestAutomaticFQTag = TRUE;
return def;
}
Setting `requestAutomaticFQTag` to `TRUE` implies setting
`useCork` to `TRUE`.
3 changes: 3 additions & 0 deletions docs/optlib.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -435,6 +435,9 @@ Example 2::
foo /tmp/input.pp /^class foo {$/;" c


NOTE: Giving a scope long flag implies setting `useCork` of the parser
to `TRUE`. See `cork API`.

Override the letter for file kind
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
(See also #317.)
Expand Down
94 changes: 64 additions & 30 deletions main/entry.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -884,7 +884,8 @@ static char* getFullQualifiedScopeNameFromCorkQueue (const tagEntryInfo * inner_
v = vStringNewInit (sep);
stringListAdd (queue, v);
}
v = vStringNewInit (escapeName (scope, FIELD_NAME));
/* TODO: scope field of SCOPE can be used for optimization. */
v = vStringNewInit (scope->name);
stringListAdd (queue, v);
kind = scope->kind;
}
Expand All @@ -904,6 +905,41 @@ static char* getFullQualifiedScopeNameFromCorkQueue (const tagEntryInfo * inner_
return vStringDeleteUnwrap (n);
}

extern void getTagScopeInformation (tagEntryInfo *const tag,
const char **kind, const char **name)
{
if (kind)
*kind = NULL;
if (name)
*name = NULL;

if (tag->extensionFields.scopeKind == NULL
&& tag->extensionFields.scopeName == NULL
&& tag->extensionFields.scopeIndex != CORK_NIL
&& TagFile.corkQueue.count > 0)
{
const tagEntryInfo * scope = NULL;
char *full_qualified_scope_name = NULL;

scope = getEntryInCorkQueue (tag->extensionFields.scopeIndex);
full_qualified_scope_name = getFullQualifiedScopeNameFromCorkQueue(scope);
Assert (full_qualified_scope_name);

/* Make the information reusable to generate full qualified entry, and xformat output*/
tag->extensionFields.scopeKind = scope->kind;
tag->extensionFields.scopeName = full_qualified_scope_name;
}

if (tag->extensionFields.scopeKind != NULL &&
tag->extensionFields.scopeName != NULL)
{
if (kind)
*kind = tag->extensionFields.scopeKind->name;
if (name)
*name = tag->extensionFields.scopeName;
}
}

static int addExtensionFields (const tagEntryInfo *const tag)
{
boolean isKindKeyEnabled = isFieldEnabled (FIELD_KIND_KEY);
Expand All @@ -926,6 +962,10 @@ static int addExtensionFields (const tagEntryInfo *const tag)
const char* separator = ";\"";
const char* const empty = "";
int length = 0;

boolean making_fq_tag = (doesInputLanguageRequestAutomaticFQTag ()
&& isXtagEnabled (XTAG_QUALIFIED_TAGS));

/* "sep" returns a value only the first time it is evaluated */
#define sep (first ? (first = FALSE, separator) : empty)

Expand All @@ -949,39 +989,24 @@ static int addExtensionFields (const tagEntryInfo *const tag)
getFieldName (FIELD_LANGUAGE),
escapeName (tag, FIELD_LANGUAGE));

if (isFieldEnabled (FIELD_SCOPE))
if (isFieldEnabled (FIELD_SCOPE) || making_fq_tag)
{
if (tag->extensionFields.scopeKind != NULL &&
tag->extensionFields.scopeName != NULL)
length += mio_printf (TagFile.fp, scopeFmt, sep,
scopeKey,
tag->extensionFields.scopeKind->name,
escapeName (tag, FIELD_SCOPE));
else if (tag->extensionFields.scopeIndex != CORK_NIL
&& TagFile.corkQueue.count > 0)
{
const tagEntryInfo * scope;
char *full_qualified_scope_name;

scope = getEntryInCorkQueue (tag->extensionFields.scopeIndex);
full_qualified_scope_name = getFullQualifiedScopeNameFromCorkQueue(scope);
Assert (full_qualified_scope_name);
length += mio_printf (TagFile.fp, scopeFmt, sep,
scopeKey,
scope->kind->name, full_qualified_scope_name);

/* TODO: Make the value pointed by full_qualified_scope_name reusable. */
eFree (full_qualified_scope_name);
}
const char* k = NULL, *v = NULL;

k = escapeName (tag, FIELD_SCOPE_KIND_LONG);
v = escapeName (tag, FIELD_SCOPE);

if (isFieldEnabled (FIELD_SCOPE) && k && v)
length += mio_printf (TagFile.fp, scopeFmt, sep, scopeKey, k, v);
}

if (isFieldEnabled (FIELD_TYPE_REF) &&
tag->extensionFields.typeRef [0] != NULL &&
tag->extensionFields.typeRef [1] != NULL)
tag->extensionFields.typeRef [0] != NULL &&
tag->extensionFields.typeRef [1] != NULL)
length += mio_printf (TagFile.fp, "%s\t%s:%s:%s", sep,
getFieldName (FIELD_TYPE_REF),
tag->extensionFields.typeRef [0],
escapeName (tag, FIELD_TYPE_REF));
getFieldName (FIELD_TYPE_REF),
tag->extensionFields.typeRef [0],
escapeName (tag, FIELD_TYPE_REF));

if (isFieldEnabled (FIELD_FILE_SCOPE) && tag->isFileScope)
length += mio_printf (TagFile.fp, "%s\t%s:", sep,
Expand Down Expand Up @@ -1372,7 +1397,16 @@ extern void uncorkTagFile(void)
return ;

for (i = 1; i < TagFile.corkQueue.count; i++)
writeTagEntry (TagFile.corkQueue.queue + i);
{
tagEntryInfo *tag = TagFile.corkQueue.queue + i;
writeTagEntry (tag);
if (doesInputLanguageRequestAutomaticFQTag ()
&& isXtagEnabled (XTAG_QUALIFIED_TAGS)
&& tag->extensionFields.scopeKind
&& tag->extensionFields.scopeName
&& tag->extensionFields.scopeIndex)
makeQualifiedTagEntry (tag);
}
for (i = 1; i < TagFile.corkQueue.count; i++)
clearTagEntryInQueue (TagFile.corkQueue.queue + i);

Expand Down
2 changes: 2 additions & 0 deletions main/entry.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,8 @@ extern void invalidatePatternCache(void);
extern void tagFilePosition (MIOPos *p);
extern void setTagFilePosition (MIOPos *p);
extern const char* getTagFileDirectory (void);
extern void getTagScopeInformation (tagEntryInfo *const tag,
const char **kind, const char **name);

/* Getting line associated with tag */
extern char *readLineFromBypassAnyway (vString *const vLine, const tagEntryInfo *const tag,
Expand Down
26 changes: 22 additions & 4 deletions main/field.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ static const char *renderFieldRole (const tagEntryInfo *const tag, const char *v
static const char *renderFieldRefMarker (const tagEntryInfo *const tag, const char *value, vString* b);
static const char *renderFieldExtra (const tagEntryInfo *const tag, const char *value, vString* b);
static const char *renderFieldXpath (const tagEntryInfo *const tag, const char *value, vString* b);
static const char *renderFieldScopeKindName(const tagEntryInfo *const tag, const char *value, vString* b);

#define DEFINE_FIELD_SPEC(L, N, V, H, F) \
{ \
Expand Down Expand Up @@ -113,7 +114,7 @@ static fieldSpec fieldSpecsExuberant [] = {
"Signature of routine (e.g. prototype or parameter list)",
renderFieldSignature),
DEFINE_FIELD_SPEC ('s', NULL, TRUE,
"Scope of tag definition (WARNING: this doesn't work well as a format letter)",
"Scope of tag definition (`p' can be used for printing its kind)",
renderFieldScope),
DEFINE_FIELD_SPEC ('t', "typeref", TRUE,
"Type and name of a variable or typedef",
Expand All @@ -131,14 +132,19 @@ static fieldSpec fieldSpecsUniversal [] = {
"Marker (R or D) representing whether tag is definition or reference",
renderFieldRefMarker),
DEFINE_FIELD_SPEC ('Z', "scope", FALSE,
"Include the \"scope:\" key in scope field (use s)",
NULL),
"Include the \"scope:\" key in scope field (use s) in tags output, scope name in xref output",
/* Following renderer is for handling --_xformat=%{scope};
and is not for tags output. */
renderFieldScope),
DEFINE_FIELD_SPEC ('E', "extra", FALSE,
"Extra tag type information",
renderFieldExtra),
DEFINE_FIELD_SPEC ('x', "xpath", FALSE,
"xpath for the tag",
renderFieldXpath),
DEFINE_FIELD_SPEC ('p', "scopeKind", TRUE,
"Kind of scope as full name",
renderFieldScopeKindName),
};


Expand Down Expand Up @@ -438,7 +444,10 @@ static const char *renderFieldSignature (const tagEntryInfo *const tag, const ch

static const char *renderFieldScope (const tagEntryInfo *const tag, const char *value __unused__, vString* b)
{
return renderEscapedName (WITH_DEFUALT_VALUE(tag->extensionFields.scopeName), tag, b);
const char* scope;

getTagScopeInformation ((tagEntryInfo *const)tag, NULL, &scope);
return scope? renderEscapedName (scope, tag, b): NULL;
}

static const char *renderFieldInherits (const tagEntryInfo *const tag, const char *value __unused__, vString* b)
Expand Down Expand Up @@ -677,6 +686,15 @@ static const char *renderFieldXpath (const tagEntryInfo *const tag,
return NULL;
}

static const char *renderFieldScopeKindName(const tagEntryInfo *const tag,
const char *value,
vString* b)
{
const char* kind;

getTagScopeInformation ((tagEntryInfo *const)tag, &kind, NULL);
return kind? renderAsIs (b, kind): NULL;
}

extern boolean isFieldEnabled (fieldType type)
{
Expand Down
3 changes: 2 additions & 1 deletion main/field.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,8 @@ typedef enum eFieldType { /* extension field content control */
FIELD_SCOPE_KEY,
FIELD_EXTRA,
FIELD_XPATH,
FIELD_BUILTIN_LAST = FIELD_XPATH,
FIELD_SCOPE_KIND_LONG,
FIELD_BUILTIN_LAST = FIELD_SCOPE_KIND_LONG,
} fieldType ;

struct sFieldDesc;
Expand Down
3 changes: 3 additions & 0 deletions main/fmt.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,9 @@ static int printTagField (fmtSpec* fspec, MIO* fp, const tagEntryInfo * tag)
else
str = renderFieldEscaped (ftype, tag, NO_PARSER_FIELD);

if (str == NULL)
str = "";

if (width < 0)
i = mio_printf (fp, "%-*s", -1 * width, str);
else if (width > 0)
Expand Down
9 changes: 8 additions & 1 deletion main/parse.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,12 @@ extern boolean doesLanguageAllowNullTag (const langType language)
return LanguageTable [language]->allowNullTag;
}

extern boolean doesLanguageRequestAutomaticFQTag (const langType language)
{
Assert (0 <= language && language < (int) LanguageCount);
return LanguageTable [language]->requestAutomaticFQTag;
}

extern const char *getLanguageName (const langType language)
{
const char* result;
Expand Down Expand Up @@ -1391,7 +1397,8 @@ extern void initializeParser (langType lang)
installTagXpathTable (lang);
installFieldSpec (lang);

if (hasScopeActionInRegex (lang))
if (hasScopeActionInRegex (lang)
|| parser->requestAutomaticFQTag)
parser->useCork = TRUE;

if ((parser->initialize != NULL) && (parser->initialized == FALSE))
Expand Down
2 changes: 2 additions & 0 deletions main/parse.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,7 @@ typedef struct {
unsigned int method; /* See PARSE__... definitions above */
boolean useCork;
boolean allowNullTag;
boolean requestAutomaticFQTag;
const tagRegexTable *tagRegexTable;
unsigned int tagRegexCount;
const keywordTable *keywordTable;
Expand Down Expand Up @@ -199,6 +200,7 @@ extern int makeSimpleRefTag (const vString* const name, kindOption* const kinds,
extern parserDefinition* parserNew (const char* name);
extern parserDefinition* parserNewFull (const char* name, char fileKind);
extern boolean doesLanguageAllowNullTag (const langType language);
extern boolean doesLanguageRequestAutomaticFQTag (const langType language);
extern const char *getLanguageName (const langType language);
extern kindOption* getLanguageFileKind (const langType language);
extern langType getNamedLanguage (const char *const name, size_t len);
Expand Down
Loading

0 comments on commit 3c9a4fe

Please sign in to comment.