1.18 COPY : Loading records with lesser number of data columns. Using TRAILING NULLCOLS option

COPY : Loading records with lesser number of data columns. Using TRAILING NULLCOLS option

Following file shows that the 2nd record in the file does not enough number of columns.

<> /home/sukul1 $ cat employeedet1.dat

SUKUL||2010-12-20|10000

RUTUJA|25

Following shows the error we get when we run the copy

sukul1=> COPY USER_30_DAY_TABLES.EMPLOYEE_1

sukul1-> (

sukul1(> NAME,

sukul1(> AGE,

sukul1(> JOINING_DATE FILLER VARCHAR(10),

sukul1(> JOINING_YEAR AS TO_NUMBER(TO_CHAR(TO_DATE(JOINING_DATE,'YYYY-MM-DD'),'YYYY')),

sukul1(> SALARY

sukul1(> )

sukul1-> FROM LOCAL '/home/sukul1/employeedet1.dat'

sukul1-> DELIMITER '|'

sukul1-> ABORT ON ERROR

sukul1-> DIRECT;

ERROR 2035:  COPY: Input record 2 has been rejected (Too few columns found)

sukul1=>

sukul1=> select * from USER_30_DAY_TABLES.EMPLOYEE_1;

 NAME | AGE | JOINING_YEAR | SALARY

------+-----+--------------+--------

(0 rows)

In above scenario we had chosen the "AbORT ON ERROR" option. That caused the entire statement to be rolled back and even the correct row was rolled back.

In Such cases we could use the TRAILING NULLCOLS option to load NULLS for the missing columns.

Following COPy statement uses the TRAILING NULLCOLS option.

Note that it does not reject the row and loads NULL's for missing columns.

sukul1=> COPY USER_30_DAY_TABLES.EMPLOYEE_1

sukul1-> (

sukul1(> NAME,

sukul1(> AGE,

sukul1(> JOINING_DATE FILLER VARCHAR(10),

sukul1(> JOINING_YEAR AS TO_NUMBER(TO_CHAR(TO_DATE(JOINING_DATE,'YYYY-MM-DD'),'YYYY')),

sukul1(> SALARY

sukul1(> )

sukul1-> FROM LOCAL '/home/sukul1/employeedet1.dat'

sukul1-> DELIMITER '|'

sukul1-> TRAILING NULLCOLS

sukul1-> ABORT ON ERROR

sukul1-> DIRECT;

 Rows Loaded

-------------

           2

(1 row)

sukul1=> select * from USER_30_DAY_TABLES.EMPLOYEE_1;

  NAME  | AGE | JOINING_YEAR | SALARY

--------+-----+--------------+--------

 RUTUJA |  25 |              |

 SUKUL  |     |         2010 |  10000

(2 rows)

Caution: This option cannot be used if the columns are defined as NOT NULL.

COPY would try to insert NULL's in NOT null columns and reject the row.

1.14 COPY : Specifying Load Metadata

COPY : Specifying Load Metadata

In addition to choosing a parser option, COPY supports other options to determine how to handle raw data.

These options are considered load metadata, and we can specify metadata options in different parts of the COPY statement.

Following are the places where we can specify the load metadata:

1. As a Column or Expression.

2. As a Column Option.

3. As a FROM level Option.

Following table shows Which option can be specified at what level.

Metadata Option	As a Column or Expression Option	As a  COLUMN OPTION	As a FROM  Level Option
DELIMITER	Y	Y	Y
ENCLOSED BY	Y	Y	Y
ESCAPE AS	Y	Y	Y
NULL	Y	Y	Y
TRIM	Y		Y
RECORD TERMINATOR			Y
SKIP			Y
SKIP BYTES			Y (Fixed-width only)
TRAILING NULLCOLS			Y

Notice that the NULL, ESCAPE,ENCLOSED and DELIMITER (NEED) metadata options can be specified at all the 3 levels.

The following precedence rules apply to all data loads:

• All column-level parameters override statement-level parameters.

• COPY uses the statement-level parameter if you do not specify a column-level parameter.

• COPY uses the default metadata values for the DELIMITER, ENCLOSED BY, ESCAPE AS, and NULL options if you do not specify them at either the statement- or column-level.