Print PDF Tables as Line Items no longer working in 7.12 and 8.1

acatt · 19 October 2022 21:38

We use WYSIWYG fields in our Products.
If you have a table in that WYSIWYG field, it no longer prints columns correctly.
If your table is:
A B
C D
It will print the A and B on top of each other, and then next line/row has the C overwritten by the D.
It works correctly in 7.11, but 7.12 and 8.1 has this odd behavior.
A table directly inserted into the PDF template will print correctly, but if the table is in a Line Item, then it does not work.

acatt · 2 December 2022 21:32

I guess this was caused by the switch from mPDF to Tcpdf.
I’m happy to pay someone who can help… How do we create bug bounties? $200 to whoever can help me!

acatt · 28 June 2023 15:59

I guess I should provide more information.
Here is my Product (Specification is a WYSIWYG field):

But when the Quote is printed with TCPDF, I get:
quote

This is preventing us from upgrading.
I am willing to pay to have this bug fixed and then donate the fix back to SuiteCRM.
Can someone quote me a fix?

pgr · 28 June 2023 19:27

And how exactly does the WYSIWYG field rendering a PDF template in the first place? How can I set that up here, in a similar way to what you have?

acatt · 28 June 2023 19:46

Sorry, I guess I did not include all the info.
I create a PDF Template:

Then create a quote with the item:

Now “Print as PDF” generates the PDF Quote.

pgr · 29 June 2023 05:58

That’s still not telling my how the field does it’s magic. I never heard of a field outputting PDF content, there’s probably something there (custom code? An add-on) doing that, no?

acatt · 29 June 2023 13:20

Hi PGR, thanks for your time and effort to understand. I’m still surprised this is not a common problem.

The field does not output PDF. The quote outputs PDF when you select “Print to PDF” from the drop-down.

The Product field (Specifications = $aos_products_specs_c) is a standard WYSIWYG field we added in which I’ve entered a simple table. This Product (with the specifications field in it) is then included in a standard quote. Then “Print to PDF” the quote.

This worked fine with mPDF, but not with TCPDF.
I’ve looked at the TCPDF documentation and it looks like it should handle tables fine.
So I think there was a change in the way line items are sent to TCPDF vs the way they are sent to mPDF.

Note: I mention ‘line items’. If you insert a table directly into the Quote template, it works. But when the table comes from a product (via line items) it works in mPDF but not in TCPDF.

pgr · 30 June 2023 10:12

Ok, I understand now. I was looking at your description and thinking of something else, sorry.

I can’t find time to work on this, unfortunately. Maybe someone else wants to take the offer a few posts above?

pstevens · 1 July 2023 16:52

I’ve tested this and it works:

Go to admin, PDF Settings and change it back to legacy. Tables in a WYSIWYG field render properly.

(You are right, the don’t render properly with the new engine)

pstevens · 1 July 2023 17:19

This is what is actually output, it looks correct, and given that it works with legacy, I don’t think it’s an output or a parsing thing:

 [aos_quotes_test_wysiwyg_c] => <table style="border-collapse:collapse;width:100%;" border="1">
<tbody>
<tr>
<td style="width:23.558%;">Title1</td>
<td style="width:23.558%;">Title2</td>
<td style="width:23.558%;">Title3</td>
<td style="width:23.558%;">Title4</td>
</tr>
<tr>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
</tr>
<tr>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
</tr>
</tbody>
</table>

pstevens · 1 July 2023 17:35

I suspect the problem might be here. The is some discussion online about UTF-8 formatting and if it’s not formatted in UTF-8 properly, it breaks the table. However, this looks like it gets formatted as UTF-8.

elseif ($field_def['type'] == 'wysiwyg') {
                    $repl_arr[$key . "_" . $field_def['name']] = html_entity_decode($focus->$field_def['name'],
                        ENT_COMPAT, 'UTF-8');
                    $repl_arr[$key . "_" . $fieldName] = html_entity_decode($focus->{$fieldName},
                        ENT_COMPAT, 'UTF-8');

\modules\AOS_PDF_Templates\templateParser.php

acatt · 5 July 2023 20:50

Hello @pstevens, Thanks for your help!

Unfortunately, I have a fresh install, so legacy “mPDF” is not an option.

When I change those two lines in \templateParser.php to an ISO encoding (or just replace ‘UTF-8’ with junk), nothing changes… (after a reboot/repair). Maybe this parameter is not getting passed correctly?

Also, if I edit my PDF template to:

The output is:

And if I add the WYSIWYG field to the description field in SuiteCRM Sample Quote, again the “ab” and “12” are overwrite each other.

So tables are processed correctly from the template. The problem is how the template is importing the “Line Items” data??? But again, it works in mPDF, but not TCPDF.

FYI: I just launched a new 8.3.0 and it does the same.

pstevens · 5 July 2023 21:08

Wonder what would happen if you you installed the legacy MPDF?
Wondering if you can just install the folder? I’ve got the latest version and still have access to the MPDF.

I tried encoding the value of the wysiwyg field to UTF-8 before it passed through the parser. That didn’t help either. Think this is a bug with TCPDF and not a suitecrm issue. I see there are a few bug reports for similar type problems on Github for TCPDF in the TCPDF repository.

acatt · 17 November 2023 15:15

OK, I’m up to a $400 reward to anyone who fixes this problem.

Summary:

mPDF works, Tcpdf does not.
Tables in a PDF template are OK. Tables in a product line-item are not.

pstevens: given item 2 above, I still think it is a SuiteCRM problem rather than a Tcpdf problem…

(Note: still an issue in version 7.14.2)

pgr · 17 November 2023 17:26

I am tempted, not just by your reward, but also by my curiosity. I’m still busy but I have some hope of fixing this relatively quickly.

But before committing to this I would need to make sure I can reproduce it here in my dev environment. I’m running 8.4.1.

Can you please give me a complete list of steps to re-create on a clean system. I mean everything, from the creation of Quotes, fields, templates, etc. I would love to have this very detailed, as in “click here”, “go there”, “paste this”, etc. I want to have a case that is exactly like the one you’re using in your tests there.

abuzarfaris · 17 November 2023 17:37

Hey
So if you don’t have any new lines in your WYSIWYG field you will not have formatting issue
e.g

<table style="border-collapse:collapse;width:100%;" border="1">
<tbody>
<tr>
<td style="width:23.558%;">Title1</td>
<td style="width:23.558%;">Title2</td>
<td style="width:23.558%;">Title3</td>
<td style="width:23.558%;">Title4</td>
</tr>
<tr>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
</tr>
<tr>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
</tr>
</tbody>
</table>

the above will cause formatting issue
But the same without new lines

<table style="border-collapse:collapse;width:100%;" border="1"><tbody><tr><td style="width:23.558%;">Title1</td><td style="width:23.558%;">Title2</td><td style="width:23.558%;">Title3</td><td style="width:23.558%;">Title4</td></tr><tr><td style="width:23.558%;">test</td><td style="width:23.558%;">test</td><td style="width:23.558%;">test</td><td style="width:23.558%;">test</td></tr><tr><td style="width:23.558%;">test</td><td style="width:23.558%;">test</td><td style="width:23.558%;">test</td><td style="width:23.558%;">test</td></tr></tbody></table>

will not give you formatting issues

Now that the issue is identified the fix

don’t add new lines WYSIWYG
Add a before save logic hook that replaces new lines with a space in the WYSIWYG field
Have a look at the modules/AOS_PDF_Templates/generatePdf.php line 138
replace

$printable = str_replace("\n", "<br />", (string) $converted);

with

$printable = $converted;

But this might have wider consequences

My suggestion is 2nd option

After fixing

The logic hook will look like
custom/Extension/modules/AOS_Products/Ext/Logichooks/cleanWYSIWYG.php

<?php
$hook_array['before_save'][] = Array(1, 'Clean WYSIWYG', 'custom/modules/AOS_Products/cleanWYSIWYG.php','cleanWYSIWYG', 'do_clean');

custom/modules/AOS_Products/cleanWYSIWYG.php

<?php

class cleanWYSIWYG
{

    public function do_clean($bean){
        $bean->specs_c=str_replace("\r\n",' ',$bean->specs_c);
    }
}

acatt · 19 November 2023 19:42

Fantastic! Thanks Abuzar! I owe you $400 - let me know the best way to send. (Also I’d like to send a donation to @pgr and @pstevens for you help in this and all the other threads - again, let me know the best way).

So the problem is almost solved.

What is the best way to ‘fix’ this natively in future versions. This work-around to remove the new-lines works, but it’s not really ‘fixing’ the problem… perhaps that’s easy for someone now that we know the cause?
The logic hook is not working correctly for me. It removes some of the new-lines, but not all. In your example @abuzarfaris, the database now shows:

<table style="border-collapse:collapse;width:100%;" border="1"><tbody><tr><td style="width:23.558%;">Title1</td>
<td style="width:23.558%;">Title2</td>
<td style="width:23.558%;">Title3</td>
<td style="width:23.558%;">Title4</td>
</tr><tr><td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
</tr><tr><td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
<td style="width:23.558%;">test</td>
</tr></tbody></table>

abuzarfaris · 20 November 2023 05:11

Hey @acatt
You can modify the logichook to

<?php

class cleanWYSIWYG
{

    public function do_clean($bean){
        $bean->specs_c=str_replace("\r\n",' ',$bean->specs_c);
    }
}

And it will remove all the new lines

To fix the issue completely we will have to evaluate the usage of

 $printable = str_replace("\n", "<br />", (string) $converted);

probably instead of the entire document it should be applied to each field’s value before their placeholder is replaced and not applied in the case of a WYSIWYG field

pgr · 20 November 2023 11:35

Since you ask

On the right-hand column, you just need to select either a one-time contribution (and your chosen amount) or a regular monthly sponsorship.

Thanks, I appreciate it.