I don’t know if this is some bug with EE, but how do I parse new lines while using Markdown in a channel text field or comments?
Example:
If you write the following comment:
This is some text.
This is a new line.
If you use the {comment}
or channel tag it gets rendered in a template as the following:
This is some text. This is a new line.
But if you go to the EE control panel and try to edit the comment or channel field, it does show correctly as new lines.
The markdown add-on manual has the following documentation tags but they make no difference:
{exp:markdown}
Text to be **parsed**.
{/exp:markdown}
What are those tags for is a good question.
Strangely, in the control panel comments list, it also shows as a single line in the dashboard. But when you actually load the editor to modify the text, it shows fine. This means the control panel is applying some CSS or JS style or something extra to properly detect the new lines
And as everyone can see already, this current ExpressionEngine forum also parses new lines correctly. Otherwise you would be reading what I just wrote as a complete one single line.
Is this a bug while trying to parse markdown on templates? I can’t find anything in the documentation that explains I have to pass a style or render Markdown tags in any specific way.
As a side note, all markdown is actually rendered properly with a template EXCEPT new lines such as h tags or code, or quotes. All except new lines. Putting 2 spaces behind seems to work, but this is not the way users write.
In the database they also stored as such (one line, but the space is there) which means I have to apply something in the template like the EE control panel does in order to show new lines properly.
This is extremely annoying because it makes markdown in text fields or comments not usable as it is not formatting new lines properly.
After tons of researching and testing, I think this is indeed a bug with Expression Engine. The reason it shows correctly in the control panel is because its a text area element.
I have now tested different markdown editors and all of them will output this: My text New line
< p >My text< br >New line< / p >
Even the preview on a editor like editor.md shows this correctly before submitting a comment.
But when you preview a comment or submit one. Expression Engine actually parses the same text as:
< p >My text New line< / p >
Does someone see the bug here? Expression Engine is not adding the proper break tag. I’m actually surprised this bug even exists in 2022 since Markdown is so popular among developers and the only options for text fields are either plain, html or markdown which seems weird this was not further tested. Setting the EE options to auto line of course adds this but then you can’t render markdown. It seems the fix would to add the auto line formatting function to the markdown to fix this. This means use auto line + markdown.
Note: The double-spacing I added between HTML tags is intentional because otherwise the forum post will render the HTML instead of showing the code.
I think all you need to do is leave a blank line between your two lines… so
This is some text.
This is a new line.
I don’t use the comment feature, so I cannot comment on whether or not this actually works there, but in other parts of EE that apparently use the same markdown calls that works to give two blocks of text (i.e. two HTML paragraph constructs).
I have also got examples of where a br tag inserted into text that is then interpreted using markdown being translated into a newline (e.g. the Readme file for JCOGS Image).
Possibly the issue is that the comment module does something like strip some HTML tags before rendering the text… someone who is more familiar with comment maybe can provide more insight on that point. Certainly if this forum is any guide that is something that EE seems to do - try putting an HTML tag into your text here and it gets stripped before the text is rendered.
As for your wider deprecation of the Markdown processing provided by EE, I think your suggestion that there is something that can be improved is hard to justify. The Markdown features in EE are ultimately provided by use of the php Markdown library - a highly respected and widely used solution. I have no idea why it was chosen, but in comparison with other popular solutions it seems to be fast and well specified; possibly it was the best solution back when EE2/3 was being specified…(the library has been in development since 2007).
Possibly given how EE uses Markdown there might be a case for switching to use CommonMark library - as it seems to be more fully featured and is the Markdown library used by Laravel (if that is any consideration).
The issue exists not only with comments but also text fields channels that are set to use Markdown. The obvious fix is to have both auto line and markdown working together, but you can only select one setting on a field, its either auto line or markdown but not both.
You can actually create the new line with 2 spaces after the end, which is a Markdown format.
The problem is that I’m aware of this. But people just typing in a website that don’t know markdown will just press Enter once, not twice, and it will not create a new line properly. Everything will be displayed as one single line which is very hard to read and ugly.
Every markdown site I used does this automatically to avoid the issue. And enabling HTML for those users is dangerous. HTML should only be enabled for trusted users. Hence, in comments and public channels, it makes sense to use Markdown, which is still better than just plain text alone without any formatting.
Maybe there is some JS that can automatically create a new line when someone hits enter while typing, and I can use in the text box. Possibly that can force a new line automatically on typing.
That could work, but you will not be able to have any text such as this displayed:
Text
Text
Text
It will always look (forced) like the following: text
text
text
text
Which is better than showing everything as a single line, but the proper fix is for EE to just add a BR tag at the end of a line like everyone does.
> … the proper fix is for EE to just add a BR tag at the end of a line like everyone does.
I guess your mileage may vary - it might be that every markdown service you use does that, but it is certainly not true that every markdown service does.
Here is a block of text from the Readme.md file associated with JCOGS Image.
### 1.2.6 (21 Feburary 2022)
Add Auto Sharpen filter
Add Rounded Corners parameter
Add Reflection parameter
Add Sepia filter (two versions of)
Add border support for masked shapes / rounded corners
Add option to limit max dimensions for processing of image
Add looking for remote images in CE Image remote cache if all else fails
Add user selectable cache filename separator
Improved: sharpen filter now uses unsharp mask (matching CE Image)
Improved: filename processing avoids separator clashes
Improved: very much faster image validation
Improved: better CP layout
Improved: image format selection logic when chosen format not supported
Improved: error trapping / reporting (421 Savepath issue)
Improved: processing time reporting
Improved: parameter validation for rotate and flip operations
Improved: manipulation parse sequence more accurately follows that used by CE Image
Improved: moved some functions to Image Utilities Class
Bug fixes for
- calculation of watermark repeat offsets
- colour validation bug (rba contains non-numerical values)
- colour validation bug (validation of three character colours -> black)
- initial dimension calculation (use round not int)
- php8.1 compatibility
- rotated image dimension calculation
If I render this using either EE’s internal markdown service or the markdown preview in VSCode I get this…
1.2.6 (21 Feburary 2022)
Add Auto Sharpen filter Add Rounded Corners parameter Add Reflection parameter Add Sepia filter (two versions of) Add border support for masked shapes / rounded corners Add option to limit max dimensions for processing of image Add looking for remote images in CE Image remote cache if all else fails Add user selectable cache filename separator Improved: sharpen filter now uses unsharp mask (matching CE Image) Improved: filename processing avoids separator clashes Improved: very much faster image validation Improved: better CP layout Improved: image format selection logic when chosen format not supported Improved: error trapping / reporting (421 Savepath issue) Improved: processing time reporting Improved: parameter validation for rotate and flip operations Improved: manipulation parse sequence more accurately follows that used by CE Image Improved: moved some functions to Image Utilities Class
Bug fixes for
calculation of watermark repeat offsets
colour validation bug (rba contains non-numerical values)
colour validation bug (validation of three character colours -> black)
initial dimension calculation (use round not int)
php8.1 compatibility
rotated image dimension calculation
The same text rendered using the Marked 2 app on macOS comes up as shown in the attached image.
The VSCode version is the ‘correct’ markdown rendering… Marked 2 only generates the text with lines because it has a default option to ‘retain line breaks within paragraphs’ - if I uncheck this I get the same output from Marked 2 as from the VSCode preview.
If what you want is a similar option added to EE’s markdown processing then put in a feature request … but in the interim I think you just have to accept that what EE is doing is what it should…
HTH
I agree, it’s not strict markdown, but I yet have to find like in your example a markdown renderer that does not do this. Most I tested, even browsers show them fine. I tested my IDE, Firefox, Edge, my File Explorer, all of them show new lines correctly even if they do not contain double-spacing at the end. I tested multiple README markdown files from developers, just randomly from GitHub and tons of them do not even have the double spaces at the back. Most actually don’t and some only have them on some lines. Nobody would ship MD files if this was strictly true. It would make it hard to read with huge horizontal scrolls on every file.
On a website, it makes markdown almost unusable for regular people that type in a text box. Let’s face it, people will not actually enter two spaces after each line they write or hit the enter key twice. Claiming this is normal is like a text editor showing the bom bytes encoding when opening a file to the user.
Developers might properly create their README MD files, but we are talking here about a CMS software, like ExpressionEngine, markdown on text fields are supposed to be used to write and show text and users are supposed to type content. When used in a text field in a channel or comment, we can’t seriously expect someone to double-space each line.
How many websites do you see that instructs someone to type two spaces after each new line they write? Even this EE forum we are typing now does it properly. I surely never double space the end of each line I’m writing this current text, and this forum supports markdown as well.
If you check the source code text of my current reply, while it’s not adding break lines it does put each new line on its own HTML < p > tag which has the same effect to render a new line. I’m ok with that as well as long as I can render each line separately in the content.
I don’t really understand why you are getting so agitated about this. Perhaps you are hoping that someone else who will read this will do something to make you feel better; unfortunately I do not have any influence with anyone - I was just trying to help.
The basic points are still the same despite your venting of your frustrations
I think your original question has been answered, if you have further points to make please leave them here, but I will not be replying to them: I think I have passed on all the useful knowledge on this topic that I can.
I’m not agitated. I have no idea how you even came to that conclusion. And who exactly I’m berating by asking a question? I have no idea why you go into personal details when we are just discussing the markdown approach used in the software. I was not expecting you to help me, you are not forced to reply or comment in the forums, if my post bothers you, you are free to click away.
As for the implementation in EE I don’t argue that. I started this debate in order to see if there was a way to properly format new lines when using the Markdown option in a text field. If we have tags like {comment} there are supposed to be used with a template. And my question was if there was a way to properly render new lines like most markdown viewers, readers and parsers do. And before you go into, that is not official markdown, I have installed 2 different WYSIWYG markdown editors on websites, and both do the same when you preview markdown text while writing. I even use a markdown app Joplin for internal notes on my systems besides OneNote, and again it also does the same. If you are that picky about this standard, 99.9% of markdown pieces of software do it. I yet have never seen one markdown render or viewer that does not, Expression Engine is the first in years. And I’m 100% fine they are following the markdown standard, but they also seem to make exceptions. This forums already proves you they are not following the implementation here either. I’m sure you have not double-spaced every line in your reply, and neither do even the EE developers when they type here.
I have no idea why you find it strange if someone is asking for the same visual implementation when rendering markdown using Expression Engine when Expression Engine already does this, both inside the control panel, in the forums and last time I remember even the Wiki modules does it when you set it to markdown because I used it for years.
As for the implementation in EE is normal. It’s bugged. Try using the plugin tags:
{exp:markdown}
Text to be parsed.
{/exp:markdown}
Like this: {exp:markdown}Text to be parsed.{/exp:markdown}
And watch what happens. The first one works fine, the second tries to parse your text as Markdown code with a broken < p > HTML tag that is not closed afterwards. All I did was to use the same tag in one line instead of 3, and it breaks the rendering on a template.
I never argued EE to make its own Markdown implementation. Like many other people posting here, I was just trying to see if there was an approach people are using when it comes to rendering text fields with comments or channels that are set to markdown, why else would I have posted this in the category How do I?….
If there is none, I will implement my own and problem solved. I can modify the EE Markdown text option to auto break lines or either create a JavaScript code that automatically adds new lines while someone ends a line in a text box directly on a template, or maybe just use CSS styling. Not a big deal. Either way, thanks for sharing.
As for, my original question has been answered. It has not. The title is “How to parse markdown new lines?”, I did not ask how to type new lines in Markdown. I used the word parse for a reason.
If there is no way to parse in a template new lines used in Markdown, as mentioned, I will just make sure they are set before hitting the POST submit button in a template with my own implementation. The visual editor I use already does this on the site, I will just modify it to use the preview in the POST instead of the actual value in a comment box. Since the preview is already adding the proper HTML tags for new lines.
Possible solutions for anyone else looking:
https://stackoverflow.com/questions/15806062/can-codemirror-show-markdown-line-breaks
Note: I’m actually wrong about my assumptions below, so please look at my next post instead. I’m keeping this one for completeness, as it perfectly shows the rabbit hole one can get caught in with regards to markdown directly in Templates 😊
No, vw000 is right, there is a bug in EE’s markdown implementation. Turns out it all has to do with code indentation!
If I create a new template group, and then – in its blank, new index template – paste the example code from the Markdown Add-On’s manual page:
{exp:markdown}
Text to be **parsed**.
{/exp:markdown}
…it renders perfectly well.
However, if the {exp:markdown}
tag pair or the text, is indented with more than two spaces in the template code, it renders as something the parser here on the forums won’t even show, but it’s exactly as vw000 described above 😊
This cannot possibly be the intention, as it’s very normal to indent things when coding a template … one would think 😆
Now, you can just not indent that part of the template, or you can create a new channel or field to hold your markdown. But I just needed a very simple About page for a site, with some text written in Markdown displayed. Just as simple as can be, so I just pasted it in the template directly (since it’s a site for myself so I can easily edit the code if needed).
That’s how I stumbled into this rabbit hole, and since the manual does say “To use this plugin wrap any text in this text pair.” - which, granted, is also wrong, they meant “tag pair” - I assumed it would just work. Perhaps it should say “…but do not indent anything, for the love of god – or you’ll spend an hour investigating” 😉
Other things are also very unpredictable when using the {exp:markdown}
tags directly in a template, so it’s really only useable from a markdown-formatted field in an entry, as it stands now. That, however, works like a charm. I’m using it for all content on one of my sites, so the lesson would be to keep it in entries.
Oh, and perhaps just remove the option to use the tag pair directly, or advice against it in the docs. If I get the time I’ll create an issue on GitHub, but since I’m an hour behind it won’t be now 😉
Cheers and love,
Thomas
PS: I posted this a while ago, that shows another approach using Zero-MD instead: https://greycells.net/news/integrate-markdown-content-with-expressionengine
Oops, this is embarrassing…
It seems many markdown parsers (correctly) will interpret a space or spaces, as the start of a code block. So EE’s is doing what it seems it should. It’s just very unexpected and confusing. I apologize that I blamed EE for this behavior!
Anyways, if anyone wants more control over markdown in EE Templates, you can look at this article I wrote in 2021:
Integrate Markdown content with ExpressionEngine
Then take a look at Zero-MD here. And especially under Advanced Usage → Dedent inline markdown. This will attempt to allow indentation of ones markdown directly in the HTML, and therefore also in EE Templates.
Quote: You can opt-in to apply a dedent function onto the inline markdown. The function tries to remove the leading whitespace that would otherwise be incorrectly interpreted as a code block by the markdown parser.
Zero-MD is an excellent markdown to HTML parser/converter, that I use extensively on Greycells.net.
Sorry again for the confusion everyone!
Cheers,
Thomas
Force Line Breaks within Paragraphs: Use two spaces at the end of a line for a visual break without starting a new paragraph. fnf
@geometry dash lite: Recognizing Markdown’s organization principles is necessary to write programs more quickly because of the syntax of new lines. The fact that the comments module removes some HTML tags before displaying the content could be the cause of the issue. If you would like to indent inline markup, you may do so.
Packet Tide owns and develops ExpressionEngine. © Packet Tide, All Rights Reserved.