Skip to content

Added support for data samples using the "records" keyword - #15

Open
TheStaticTurtle wants to merge 1 commit into
LiamClarkeNZ:mainfrom
TheStaticTurtle:feature-records
Open

Added support for data samples using the "records" keyword#15
TheStaticTurtle wants to merge 1 commit into
LiamClarkeNZ:mainfrom
TheStaticTurtle:feature-records

Conversation

@TheStaticTurtle

Copy link
Copy Markdown

Hi,

This PR adds support for data samples : https://dbml.dbdiagram.io/docs#data-sample

Records allows you to define sample data for your tables directly in DBML. This is useful for documentation, testing, and providing example data for your database schema.

For some reason, my environment refuses to start tests, but I can successfully build it and use it inside the IDE:
image

Note that this change only provides the base syntax part, it does not check that columns actually exist or that the number of elements in records_column_list matches the number of elements in records_row

@LiamClarkeNZ

Copy link
Copy Markdown
Owner

Heads up - did a local test run to backfill the missing Records.txt expected PSI fixture.

ParsingTestCase needs a paired .txt file for each .dbml test fixture, which wasn't included in the PR. I fetched your branch, ran the test with -Didea.tests.overwrite.data=true to have the harness generate the baseline, and captured the output. Drop it at src/test/testData/parser/Records.txt in your branch and push.

Generated Records.txt (277 lines)
DBML File(0,647)
  PsiComment(LINE_COMMENT)('// Outside table definition')(0,27)
  PsiElement(new line)('\n')(27,28)
  DbmlTableDefinitionImpl(TABLE_DEFINITION)(28,88)
    PsiElement('Table')('Table')(28,33)
    PsiWhiteSpace(' ')(33,34)
    DbmlTableNameImpl(TABLE_NAME)(34,39)
      PsiElement(identifier)('users')(34,39)
    PsiWhiteSpace(' ')(39,40)
    PsiElement('{')('{')(40,41)
    PsiElement(new line)('\n')(41,42)
    PsiWhiteSpace('  ')(42,44)
    DbmlColumnDefinitionImpl(COLUMN_DEFINITION)(44,55)
      PsiElement(identifier)('id')(44,46)
      PsiWhiteSpace(' ')(46,47)
      DbmlColumnDatatypeImpl(COLUMN_DATATYPE)(47,50)
        PsiElement(identifier)('int')(47,50)
      PsiWhiteSpace(' ')(50,51)
      DbmlColumnSettingsImpl(COLUMN_SETTINGS)(51,55)
        PsiElement('[')('[')(51,52)
        DbmlColumnSettingImpl(COLUMN_SETTING)(52,54)
          PsiElement('pk')('pk')(52,54)
        PsiElement(']')(']')(54,55)
    PsiElement(new line)('\n')(55,56)
    PsiWhiteSpace('  ')(56,58)
    DbmlColumnDefinitionImpl(COLUMN_DEFINITION)(58,70)
      PsiElement('name')('name')(58,62)
      PsiWhiteSpace(' ')(62,63)
      DbmlColumnDatatypeImpl(COLUMN_DATATYPE)(63,70)
        PsiElement(identifier)('varchar')(63,70)
    PsiElement(new line)('\n')(70,71)
    PsiWhiteSpace('  ')(71,73)
    DbmlColumnDefinitionImpl(COLUMN_DEFINITION)(73,86)
      PsiElement(identifier)('email')(73,78)
      PsiWhiteSpace(' ')(78,79)
      DbmlColumnDatatypeImpl(COLUMN_DATATYPE)(79,86)
        PsiElement(identifier)('varchar')(79,86)
    PsiElement(new line)('\n')(86,87)
    PsiElement('}')('}')(87,88)
  PsiElement(new line)('\n')(88,89)
  PsiElement(new line)('\n')(89,90)
  DbmlRecordsDefinitionImpl(RECORDS_DEFINITION)(90,188)
    PsiElement('Records')('records')(90,97)
    PsiWhiteSpace(' ')(97,98)
    DbmlTableNameImpl(TABLE_NAME)(98,103)
      PsiElement(identifier)('users')(98,103)
    PsiElement('(')('(')(103,104)
    PsiElement(identifier)('id')(104,106)
    PsiElement(',')(',')(106,107)
    PsiWhiteSpace(' ')(107,108)
    PsiElement('name')('name')(108,112)
    PsiElement(',')(',')(112,113)
    PsiWhiteSpace(' ')(113,114)
    PsiElement(identifier)('email')(114,119)
    PsiElement(')')(')')(119,120)
    PsiWhiteSpace(' ')(120,121)
    PsiElement('{')('{')(121,122)
    PsiElement(new line)('\n')(122,123)
    PsiWhiteSpace('  ')(123,125)
    DbmlRecordsRowImpl(RECORDS_ROW)(125,156)
      DbmlRecordsValueImpl(RECORDS_VALUE)(125,126)
        PsiElement(number)('1')(125,126)
      PsiElement(',')(',')(126,127)
      PsiWhiteSpace(' ')(127,128)
      DbmlRecordsValueImpl(RECORDS_VALUE)(128,135)
        PsiElement(string)(''Alice'')(128,135)
      PsiElement(',')(',')(135,136)
      PsiWhiteSpace(' ')(136,137)
      DbmlRecordsValueImpl(RECORDS_VALUE)(137,156)
        PsiElement(string)(''alice@example.com'')(137,156)
    PsiElement(new line)('\n')(156,157)
    PsiWhiteSpace('  ')(157,159)
    DbmlRecordsRowImpl(RECORDS_ROW)(159,186)
      DbmlRecordsValueImpl(RECORDS_VALUE)(159,160)
        PsiElement(number)('2')(159,160)
      PsiElement(',')(',')(160,161)
      PsiWhiteSpace(' ')(161,162)
      DbmlRecordsValueImpl(RECORDS_VALUE)(162,167)
        PsiElement(string)(''Bob'')(162,167)
      PsiElement(',')(',')(167,168)
      PsiWhiteSpace(' ')(168,169)
      DbmlRecordsValueImpl(RECORDS_VALUE)(169,186)
        PsiElement(string)(''bob@example.com'')(169,186)
    PsiElement(new line)('\n')(186,187)
    PsiElement('}')('}')(187,188)
  PsiElement(new line)('\n')(188,189)
  PsiElement(new line)('\n')(189,190)
  PsiComment(LINE_COMMENT)('// Inside table definition with explicit column list')(190,242)
  PsiElement(new line)('\n')(242,243)
  DbmlTableDefinitionImpl(TABLE_DEFINITION)(243,402)
    PsiElement('Table')('Table')(243,248)
    PsiWhiteSpace(' ')(248,249)
    DbmlTableNameImpl(TABLE_NAME)(249,254)
      PsiElement(identifier)('posts')(249,254)
    PsiWhiteSpace(' ')(254,255)
    PsiElement('{')('{')(255,256)
    PsiElement(new line)('\n')(256,257)
    PsiWhiteSpace('  ')(257,259)
    DbmlColumnDefinitionImpl(COLUMN_DEFINITION)(259,270)
      PsiElement(identifier)('id')(259,261)
      PsiWhiteSpace(' ')(261,262)
      DbmlColumnDatatypeImpl(COLUMN_DATATYPE)(262,265)
        PsiElement(identifier)('int')(262,265)
      PsiWhiteSpace(' ')(265,266)
      DbmlColumnSettingsImpl(COLUMN_SETTINGS)(266,270)
        PsiElement('[')('[')(266,267)
        DbmlColumnSettingImpl(COLUMN_SETTING)(267,269)
          PsiElement('pk')('pk')(267,269)
        PsiElement(']')(']')(269,270)
    PsiElement(new line)('\n')(270,271)
    PsiWhiteSpace('  ')(271,273)
    DbmlColumnDefinitionImpl(COLUMN_DEFINITION)(273,286)
      PsiElement(identifier)('title')(273,278)
      PsiWhiteSpace(' ')(278,279)
      DbmlColumnDatatypeImpl(COLUMN_DATATYPE)(279,286)
        PsiElement(identifier)('varchar')(279,286)
    PsiElement(new line)('\n')(286,287)
    PsiWhiteSpace('  ')(287,289)
    DbmlColumnDefinitionImpl(COLUMN_DEFINITION)(289,306)
      PsiElement(identifier)('published')(289,298)
      PsiWhiteSpace(' ')(298,299)
      DbmlColumnDatatypeImpl(COLUMN_DATATYPE)(299,306)
        PsiElement(identifier)('boolean')(299,306)
    PsiElement(new line)('\n')(306,307)
    PsiElement(new line)('\n')(307,308)
    PsiWhiteSpace('  ')(308,310)
    DbmlTableRecordsImpl(TABLE_RECORDS)(310,400)
      PsiElement('Records')('records')(310,317)
      PsiWhiteSpace(' ')(317,318)
      PsiElement('(')('(')(318,319)
      PsiElement(identifier)('id')(319,321)
      PsiElement(',')(',')(321,322)
      PsiWhiteSpace(' ')(322,323)
      PsiElement(identifier)('title')(323,328)
      PsiElement(',')(',')(328,329)
      PsiWhiteSpace(' ')(329,330)
      PsiElement(identifier)('published')(330,339)
      PsiElement(')')(')')(339,340)
      PsiWhiteSpace(' ')(340,341)
      PsiElement('{')('{')(341,342)
      PsiElement(new line)('\n')(342,343)
      PsiWhiteSpace('    ')(343,347)
      DbmlRecordsRowImpl(RECORDS_ROW)(347,368)
        DbmlRecordsValueImpl(RECORDS_VALUE)(347,348)
          PsiElement(number)('1')(347,348)
        PsiElement(',')(',')(348,349)
        PsiWhiteSpace(' ')(349,350)
        DbmlRecordsValueImpl(RECORDS_VALUE)(350,362)
          PsiElement(string)(''First Post'')(350,362)
        PsiElement(',')(',')(362,363)
        PsiWhiteSpace(' ')(363,364)
        DbmlRecordsValueImpl(RECORDS_VALUE)(364,368)
          PsiElement(identifier)('true')(364,368)
      PsiElement(new line)('\n')(368,369)
      PsiWhiteSpace('    ')(369,373)
      DbmlRecordsRowImpl(RECORDS_ROW)(373,396)
        DbmlRecordsValueImpl(RECORDS_VALUE)(373,374)
          PsiElement(number)('2')(373,374)
        PsiElement(',')(',')(374,375)
        PsiWhiteSpace(' ')(375,376)
        DbmlRecordsValueImpl(RECORDS_VALUE)(376,389)
          PsiElement(string)(''Second Post'')(376,389)
        PsiElement(',')(',')(389,390)
        PsiWhiteSpace(' ')(390,391)
        DbmlRecordsValueImpl(RECORDS_VALUE)(391,396)
          PsiElement(identifier)('false')(391,396)
      PsiElement(new line)('\n')(396,397)
      PsiWhiteSpace('  ')(397,399)
      PsiElement('}')('}')(399,400)
    PsiElement(new line)('\n')(400,401)
    PsiElement('}')('}')(401,402)
  PsiElement(new line)('\n')(402,403)
  PsiElement(new line)('\n')(403,404)
  PsiComment(LINE_COMMENT)('// Inside table definition with implicit column list')(404,456)
  PsiElement(new line)('\n')(456,457)
  DbmlTableDefinitionImpl(TABLE_DEFINITION)(457,647)
    PsiElement('Table')('Table')(457,462)
    PsiWhiteSpace(' ')(462,463)
    DbmlTableNameImpl(TABLE_NAME)(463,471)
      PsiElement(identifier)('comments')(463,471)
    PsiWhiteSpace(' ')(471,472)
    PsiElement('{')('{')(472,473)
    PsiElement(new line)('\n')(473,474)
    PsiWhiteSpace('  ')(474,476)
    DbmlColumnDefinitionImpl(COLUMN_DEFINITION)(476,487)
      PsiElement(identifier)('id')(476,478)
      PsiWhiteSpace(' ')(478,479)
      DbmlColumnDatatypeImpl(COLUMN_DATATYPE)(479,482)
        PsiElement(identifier)('int')(479,482)
      PsiWhiteSpace(' ')(482,483)
      DbmlColumnSettingsImpl(COLUMN_SETTINGS)(483,487)
        PsiElement('[')('[')(483,484)
        DbmlColumnSettingImpl(COLUMN_SETTING)(484,486)
          PsiElement('pk')('pk')(484,486)
        PsiElement(']')(']')(486,487)
    PsiElement(new line)('\n')(487,488)
    PsiWhiteSpace('  ')(488,490)
    DbmlColumnDefinitionImpl(COLUMN_DEFINITION)(490,519)
      PsiElement(identifier)('user_id')(490,497)
      PsiWhiteSpace(' ')(497,498)
      DbmlColumnDatatypeImpl(COLUMN_DATATYPE)(498,501)
        PsiElement(identifier)('int')(498,501)
      PsiWhiteSpace(' ')(501,502)
      DbmlColumnSettingsImpl(COLUMN_SETTINGS)(502,519)
        PsiElement('[')('[')(502,503)
        DbmlColumnSettingImpl(COLUMN_SETTING)(503,518)
          DbmlColumnInlineRefImpl(COLUMN_INLINE_REF)(503,518)
            PsiElement('Ref')('ref')(503,506)
            PsiElement(':')(':')(506,507)
            PsiWhiteSpace(' ')(507,508)
            DbmlRelationImpl(RELATION)(508,509)
              PsiElement('>')('>')(508,509)
            PsiWhiteSpace(' ')(509,510)
            DbmlRefColumnNamesImpl(REF_COLUMN_NAMES)(510,518)
              PsiElement(identifier)('users')(510,515)
              PsiElement('.')('.')(515,516)
              PsiElement(identifier)('id')(516,518)
        PsiElement(']')(']')(518,519)
    PsiElement(new line)('\n')(519,520)
    PsiWhiteSpace('  ')(520,522)
    DbmlColumnDefinitionImpl(COLUMN_DEFINITION)(522,551)
      PsiElement(identifier)('post_id')(522,529)
      PsiWhiteSpace(' ')(529,530)
      DbmlColumnDatatypeImpl(COLUMN_DATATYPE)(530,533)
        PsiElement(identifier)('int')(530,533)
      PsiWhiteSpace(' ')(533,534)
      DbmlColumnSettingsImpl(COLUMN_SETTINGS)(534,551)
        PsiElement('[')('[')(534,535)
        DbmlColumnSettingImpl(COLUMN_SETTING)(535,550)
          DbmlColumnInlineRefImpl(COLUMN_INLINE_REF)(535,550)
            PsiElement('Ref')('ref')(535,538)
            PsiElement(':')(':')(538,539)
            PsiWhiteSpace(' ')(539,540)
            DbmlRelationImpl(RELATION)(540,541)
              PsiElement('>')('>')(540,541)
            PsiWhiteSpace(' ')(541,542)
            DbmlRefColumnNamesImpl(REF_COLUMN_NAMES)(542,550)
              PsiElement(identifier)('posts')(542,547)
              PsiElement('.')('.')(547,548)
              PsiElement(identifier)('id')(548,550)
        PsiElement(']')(']')(550,551)
    PsiElement(new line)('\n')(551,552)
    PsiWhiteSpace('  ')(552,554)
    DbmlColumnDefinitionImpl(COLUMN_DEFINITION)(554,566)
      PsiElement(identifier)('title')(554,559)
      PsiWhiteSpace(' ')(559,560)
      DbmlColumnDatatypeImpl(COLUMN_DATATYPE)(560,566)
        PsiElement(identifier)('string')(560,566)
    PsiElement(new line)('\n')(566,567)
    PsiElement(new line)('\n')(567,568)
    PsiWhiteSpace('  ')(568,570)
    DbmlTableRecordsImpl(TABLE_RECORDS)(570,645)
      PsiElement('Records')('records')(570,577)
      PsiWhiteSpace(' ')(577,578)
      PsiElement('{')('{')(578,579)
      PsiElement(new line)('\n')(579,580)
      PsiWhiteSpace('    ')(580,584)
      DbmlRecordsRowImpl(RECORDS_ROW)(584,641)
        DbmlRecordsValueImpl(RECORDS_VALUE)(584,585)
          PsiElement(number)('1')(584,585)
        PsiElement(',')(',')(585,586)
        PsiWhiteSpace(' ')(586,587)
        DbmlRecordsValueImpl(RECORDS_VALUE)(587,588)
          PsiElement(number)('2')(587,588)
        PsiElement(',')(',')(588,589)
        PsiWhiteSpace(' ')(589,590)
        DbmlRecordsValueImpl(RECORDS_VALUE)(590,591)
          PsiElement(number)('1')(590,591)
        PsiElement(',')(',')(591,592)
        PsiWhiteSpace(' ')(592,593)
        DbmlRecordsValueImpl(RECORDS_VALUE)(593,641)
          PsiElement(string)(''First comment of first post by the second user'')(593,641)
      PsiElement(new line)('\n')(641,642)
      PsiWhiteSpace('  ')(642,644)
      PsiElement('}')('}')(644,645)
    PsiElement(new line)('\n')(645,646)
    PsiElement('}')('}')(646,647)

A few things I spotted while reviewing:

  1. Broken example in DbmlColorSettingsPage.kt: the added block has 0, 'user, 'user@example.com' - missing a closing quote on 'user. The settings-page preview will render it oddly. Likely meant 0, 'user', 'user@example.com'.

  2. Standalone form vs spec: the DBML docs (Data Sample) state "implicit column lists are only supported for records defined inside a table." Your current grammar shares records_config between both forms, so records users { ... } (no column list) parses at the top level - which the spec disallows. Suggest splitting the rules so the column list is required in the standalone form:

    records_definition ::= RECORDS table_name LPAREN records_column_list RPAREN LBRACE records_body RBRACE {pin=1}
    table_records ::= RECORDS records_config {pin=1}
    private records_config ::= (LPAREN records_column_list RPAREN)? LBRACE records_body RBRACE
    
  3. records_row allows empty cells and trailing commas: records_value (COMMA records_value?)* {pin=1} accepts 1, , , 3 and 1, 2, 3,. The spec is silent on this; if intentional for tolerance, fine. Otherwise records_value (COMMA records_value)* {pin=1} is stricter.

  4. RECORDS not added to identifier_ (line 84-88 of Dbml.bnf) - means tables and columns can no longer be named records. Defensible as it's now a reserved construct keyword, just worth a conscious call.

  5. Missing trailing newline in Records.dbml and Dbml.bnf - cosmetic.

The fixture above matches the current grammar as-is. If you adopt the spec fix in point 2, the standalone case in the fixture will change slightly - easiest to re-run with the overwrite flag after the grammar edit.

Thanks for the contribution - happy to open a small follow-up PR against your fork with these tweaks if that's easier than doing them yourself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants