Bug 53027

Summary: EDITING: dBase-Import-Wizard fails when trying to append DECIMAL(NUMERIC) and BOOLEAN data types
Product: LibreOffice Reporter: robert
Component: DatabaseAssignee: Not Assigned <libreoffice-bugs>
Status: NEW --- QA Contact:
Severity: major    
Priority: high CC: dr, iplaw67, libreoffice, lionel
Version: 3.6.0.4 release   
Hardware: All   
OS: All   
See Also: https://bugs.freedesktop.org/show_bug.cgi?id=47496
https://bugs.freedesktop.org/show_bug.cgi?id=32672
Whiteboard:
i915 platform: i915 features:
Attachments: Little dBase-file for testing
Description how to reproduce wrong formatting of dBase-import-wizard

Description robert 2012-08-01 07:19:58 UTC
Created attachment 65020 [details]
Little dBase-file for testing

I tried to import some dBase-tables into Base. The wizard shows me the types of the fields. So I only declared a primary-key.
Message from the wizard: "A error occured".
Then I had a look at the tables with Calc. The following description of the header were not read correct by the dBase-wizard:
FIELDNAME,N,5,0 → Wizard will create Decimal,4,0 - should be Decimal,5,0
FIELDNAME,L → Wizard will create Text(1) - should be Boolean
FIELDNAME,N,8,2 → Wizard will create Decimal,6,2 - should be Decimal,8,2

When I get the direct contact from Base to a *.dbf-file it works correct. But with this direct contact I could not create a query over more than one table.
Comment 1 robert 2012-08-01 07:21:07 UTC
Created attachment 65021 [details]
Description how to reproduce wrong formatting of dBase-import-wizard
Comment 2 robert 2012-08-02 05:41:48 UTC
Try to get a direct contact with Base to the dBase-file. When I open the table for editing the field-propertis I noticed, that the field with N,5,0 is shown as Decimal,4,0. But there are values in the table like "98765" - and if you are opening the table for seeing the data you could have a look: a number with fife-digit is shown in a field, that seems to be four-digit.
The field with "L" is shown correct as Boolean in the properties of the table.

So this isn't only a problem of the wizard. It's a problem, that Base doesent know, how to interpret the field-types of dBase.
Comment 3 Jochen 2012-09-01 17:50:28 UTC
@Robert,
I want to reproduce the bug but I´m not able to do this. Please list the single first steps concerning import dBase-tables into Base.

In comment 2 you write "When I open the table". What and how do you open? Please list the file.

In your description you write "This must be corrected" and "I have to change this". Are this two points/example the bugs?

In your description shows the first picture the settings of the dBase-file. Right?
Is there any doubt that the shown data (e.g. "SPECIES_NR,N,0" or "SYNONYM,L") can be wrong or a "fake"?
I´m asking because I will test without availability of dBase and set status to "NEW" if I can reproduce the steps and the behavior. I must trust that the information concerning dBase are right.

PS: another person (with dBase) obviously do not have time or inclination to test.
Comment 4 robert 2012-09-02 08:10:16 UTC
@Jochen,

before you could import a dBase table into Base you have to create a new database in Base. The wizard asks, when creating a new connection to a dBase-Database, for the folder, where to look for the dBase-files. Formerly all dBase-files were shown as tables in this new Base-Database.

Then create a new standalone Base-file: 
Database → Create a new database → Open the database for editing ..

Now you have two databases: 
The database, which connects to the dBase-files and shows all the tables inside a folder ending with *.dbf
The new database for importing the tables form the dBase-Database.

With a right-click of the mousebutton over a dBase-Tabele from your Base-file, which connects to the dBase-files, you could copy this table. After this action go to the new database, whre the file should be imported. Now you have to paste the file. The import-wizard appears as shown in the description https://bugs.freedesktop.org/attachment.cgi?id=65021 .

When you will import the table without changing the filed-types a popup appears: "A error occured". This happens, because there are values in the table, which could not be shown with the filed-types the wizard will create. And the field-types, the wizard will create, are not the same which were shown, when opening the *.dbf-file with LO-Calc. When you change the filed-types and length, the wizard creates, to the field-types and length, which are shown in Calc, the import works.
Comment 5 Rainer Bielefeld Retired 2012-09-02 09:43:31 UTC
Hi Robert,
your explanation starts far behind the point interesting here. We need the few mouse clicks required to reproduce the effect. Something like "Then create a new standalone Base-file: "Database → Create a new database → Open the database for editing .." does not exist in my LibO, or I can't find it, and be sure, I will not look for it, I have to do enough. It would really help if you could respect the "Step by step instruction with every mouse click and every key press" hint. Bug 43431 shows how that should be done.
Comment 6 robert 2012-09-02 10:37:04 UTC
@Rainer

When I open LO there is a welcomescreen with all the sub-programms of LO. There you have to click on "Database".
When you have started LO directly, for example with the sub-program writer, you could find the same with 
File → New → Database ...
Comment 7 Alex Thurgood 2012-09-02 11:32:45 UTC
@ALL : Robert's report is understandable, at least for me. I'm pretty certain I saw this reported on the English or French user list quite a while ago with a previous version of LO, but I will check whether I can reproduce it in a recent(ish) master build.

Alex
Comment 8 Alex Thurgood 2012-09-02 11:39:31 UTC
Ok, so if you want instructions, try this :

1) Create a first ODB database file that links to one or more DBF files in a folder.

2) Create a second database file, which is a native HSQLDB ODB container. Put nothing in it for the time being.

3) Open both ODB files.

4) In the ODB file that links to DBF tables, click on a Table, and then Ctrl-C, or right mouse button click and choose "Copy".

5) Now click on the Tables button of your second ODB file (the native HSQLDB one), and in the bottom central window, press Ctrl-V, or right mouse button click and choose "Paste"

6) As an alternative to steps 4 and 5, you should simply be able to select a table from the DBF-ODB and drag and drop it in the "Tables" lower central window of the HSQLDB-ODB.

7) The import wizard should start.
8) Accept the defaults, and allow the wizard to add a primary key ID field.

9) See whether the operation completes without errors, and if so, whether the imported data and field type definitions correspond to those defined in the DBF table.


Alex
Comment 9 Alex Thurgood 2012-09-02 11:47:16 UTC
I can confirm on Linux Mint 12 64bit, with LO 3.5.4
The error message is :

SQL Status: 22001
Error code: -124

Value too long in statement [INSERT INTO "speciesshort" ( "ID","SPECIES_NR","LETTERCODE","SHORTNAME","ABBREVIAT","NATIVENAME","AUTHOR","SYNONYM","VALID_NR") VALUES ( ?,?,?,?,?,?,?,?,?)]



Alex
Comment 10 Alex Thurgood 2012-09-02 11:50:03 UTC
If the user continues to press "Yes" in answer to the error message, it will eventually go away (once it has reached the number of parameters required), and the table will be created, but no data is inserted.


Alex
Comment 11 Alex Thurgood 2012-09-02 11:54:21 UTC
Also reproducible in Version 3.6.0.4 (Build ID: 932b512)
Comment 12 Alex Thurgood 2012-09-02 13:03:57 UTC
@Lionel : care to take a look ?

Alex
Comment 13 Rainer Bielefeld Retired 2012-09-02 14:26:37 UTC
I don't want to disturb experts' talks with such childish nonsense like a step by step instruction understandable for people who do not know yet what a HSQLDB ODB container is, but have bibisect installed on their PC ...

When I remove my name from CC that does not mean that I want to be added again.

I do not want to be added to CC of database bugs by others.
Comment 14 Alex Thurgood 2012-09-02 14:33:09 UTC
After transferring several different types of DBF table, I noticed that some succeeded and some failed. 

The problem lies in the copy of tables containing DECIMAL(NUMERIC) fields. This is what causes the error message to occur and the data append to fail, even if the table structure is created.


Alex
Comment 15 Alex Thurgood 2012-09-02 14:37:09 UTC
Tested also in master build on Mac OSX :
Version 3.7.0.0.alpha0+ (Build ID: 4deb9d4)

Reproducible.

I even managed to get LO to crash on OSX during the drag n drop maneuver, but as I couldn't then reproduce the crash, I've put that down to general instability on Mac. If I do manage to crash it again with the same thing, I'll post a different report with a trace.


Alex
Comment 16 Alex Thurgood 2012-09-02 14:40:10 UTC
Edited title to reflect situation.
Comment 17 robert 2012-09-02 15:09:25 UTC
@Alex

It isn't only a problem with DECIMAL(NUMERIC) data types. The wizard couldn't create Boolean also. The Type "L" is boolean in dBase. DBase-tables, shown in Base, would show such a boolean field. Imported tables in Base haven't any boolean-field.
Comment 18 Alex Thurgood 2012-09-02 16:26:24 UTC
Changed title again to include Boolean field types.
Comment 19 Alex Thurgood 2015-01-03 17:39:42 UTC
Adding self to CC if not already on

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.