Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2005 Forums
 Other SQL Server Topics (2005)
 Exporting XML - (with crlf)

Author  Topic 

rockmoose
SQL Natt Alfen

3279 Posts

Posted - 2007-04-02 : 08:10:14
I am exporting data via the "for xml auto,elements" clause.
The data is bcp:ed out to textfiles.

The recipients are now complaining that there are no line breaks between the tags.
i.e. they want it more "nicely formatted".

So my question is basically, is it possible to handle this request nicely from the server side?

I basically just do:

bcp "select blah,blah from tbl for xml auto,elements" queryout "D:\..." -w -T -t -r

Any way to get line breaks???


rockmoose

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2007-04-02 : 08:19:27
Try with "-c".
quote:
Performs the operation using a character data type. This option does not prompt for each field; it uses char as the storage type, without prefixes and with \t (tab character) as the field separator and \n (newline character) as the row terminator.

Strange though that "-r \n" does not work.


Peter Larsson
Helsingborg, Sweden
Go to Top of Page

rockmoose
SQL Natt Alfen

3279 Posts

Posted - 2007-04-02 : 09:10:27
I use -w, since we have unicode chars in the tables.
Also -w -c only put a linefeed after each row.

The for xml clause, only outputs one row/stream.

select * from
(select 'abc' as col1, 1 as col2 union all select 'def', 2) as thetable
for xml auto,elements

When exported it looks like this (open in notepad)
<thetable><col1>abc</col1><col2>1</col2></thetable><thetable><col1>def</col1><col2>2</col2></thetable>


If possible I would want it to look like this:
<thetable>
<col1>abc</col1>
<col2>1</col2>
</thetable>
<thetable>
<col1>def</col1>
<col2>2</col2>
</thetable>


Coming to think of it, I don't think it is possible from sql server.

Alternative ideas might be to run the files through some utility / parser that transforms them.
Any suggestions?

rockmoose
Go to Top of Page

SwePeso
Patron Saint of Lost Yaks

30421 Posts

Posted - 2007-04-02 : 09:29:03
If you replace "><" with ">" + CHAR(13) + CHAR(10) + "<" then?


Peter Larsson
Helsingborg, Sweden
Go to Top of Page

rockmoose
SQL Natt Alfen

3279 Posts

Posted - 2007-04-02 : 14:10:18
Yeah, there are some possibilities.
Just fishing for a quick alternative.

Seems to me it will be to ask the recipients to fix their line breaks themselves.
After all it is a presentation issue

If I am in a nice mood, I might help them.

rockmoose
Go to Top of Page

jezemine
Master Smack Fu Yak Hacker

2886 Posts

Posted - 2007-04-08 : 11:41:21
if you write the xml using XmlWriter, you can use the XmlWriterSettings class to specify how you want the indenting and linebreaks to work. See the Indent, IndentChars, NewLineChars properties on that class.

to use this class means you have to consume your stream by a .net app though. perhaps you are already doing so?




www.elsasoft.org
Go to Top of Page

rockmoose
SQL Natt Alfen

3279 Posts

Posted - 2007-04-08 : 12:36:47
quote:
Originally posted by jezemine

if you write the xml using XmlWriter, you can use the XmlWriterSettings class to specify how you want the indenting and linebreaks to work. See the Indent, IndentChars, NewLineChars properties on that class.

to use this class means you have to consume your stream by a .net app though. perhaps you are already doing so?


www.elsasoft.org



No just using bcp and select for xml.
Thanks for the suggestion, I'll check the docs for that. It might be just what I need.
I was thinking of just hardcoding the 12 selects like so:
select '<bla>' + bla + '</bla>' + char(10) + char(13)

rockmoose
Go to Top of Page

jezemine
Master Smack Fu Yak Hacker

2886 Posts

Posted - 2007-04-08 : 14:00:32
doing this is fragile:

select '<bla>' + bla + '</bla>' + char(10) + char(13)

because if your values contain any chars such as &, <, >, it will break your xml.


www.elsasoft.org
Go to Top of Page

rockmoose
SQL Natt Alfen

3279 Posts

Posted - 2007-04-08 : 16:43:56
So, using the XmlWriter, is this the way to go about it?
relevant code in bold.

Notice, using the XmlWriter I found no straightforward way to leverage the FOR XML clause in SQL.

XmlWriter xw = XmlWriter.Create(...);
SqlConnection = new SqlConnection("...");
SqlCommand cmd = cn.CreateCommand();
cmd.CommandText = "SELECT col1,col2 FROM tbl";
cn.Open();
SqlDataReader dr = cmd.ExecuteReader();

while(dr.Read())
{
for(i=0;i<dr.FieldCount;i++)
xw.WriteElementString(dr.GetName(i),dr[i].ToString());

}

cn.Close();



Not being a xml hacker I just have to ask, I might be missing the obvious


PS.
Some of the fields are ntext fields, could that cause some possible problem for the XmlWriter approach?

rockmoose
Go to Top of Page

jezemine
Master Smack Fu Yak Hacker

2886 Posts

Posted - 2007-04-08 : 17:43:52
Couple points:

1. always use the "using" keyword when using objects that implement IDisposable. Otherwise you run the risk of native resource leaks if you throw before Close() or Dispose() is called. It's a PITA, but that's what you have to do in C# if you want your code to be robust against leaks.
2. I would use attribute centric style here, with one attribute for each column. that way it's clear which column cells are part of the same row.
3. Check for null or else the SqlDataReader will throw if you access a null cell.
4. save the FieldCount in a local so you don't need to make a call for each row. minor point but it might make a difference if you have millions of rows.
5. added an example of how to specify indenting, encoding, etc with XmlWriterSettings


public void rockmoose()
{
XmlWriterSettings s = new XmlWriterSettings();
s.Indent = true;
s.Encoding = Encoding.Unicode;
s.OmitXmlDeclaration = false;

using (XmlWriter xw = XmlWriter.Create("C:\\foo.xml", s))
using (SqlConnection cn = new SqlConnection("..."))
using (SqlCommand cmd = cn.CreateCommand())
{
cmd.CommandText = "SELECT col1,col2 FROM tbl";
cn.Open();
using (SqlDataReader dr = cmd.ExecuteReader())
{
int fields = dr.FieldCount;
xw.WriteStartElement("root");
while (dr.Read())
{
xw.WriteStartElement("rockmoose");
for (int i = 0; i < fields; i++)
{
if (dr.IsDBNull(i)) continue;

xw.WriteAttributeString(dr.GetName(i), dr[i].ToString());
}
xw.WriteEndElement();
}
xw.WriteEndElement();
}
}
}



www.elsasoft.org
Go to Top of Page

rockmoose
SQL Natt Alfen

3279 Posts

Posted - 2007-04-08 : 18:14:01
Thanks Jez!

1. I would have (or used try finally) but omitted for brevity.
2. It's for a translation firm, and I think we will use the element approach, and have start-end tags for each row. (They are importing the final results into word).
3. Obviously :), nulls must always be handled, I was thinking of dr.IsDBNull(i) ? "" : dr[i].ToString();
4. Optimization is always good, for this case, writing xml woohahaha ;). (But yes, one must do what one can!)
5. Thanks again.

I'll post the final masterpiece here.

rockmoose
Go to Top of Page

rockmoose
SQL Natt Alfen

3279 Posts

Posted - 2007-04-09 : 20:04:55
Here is the powershell hack in it's gory glory, enjoy:
It creates two files, one for the schema metadata, and one for the actual exported data.
The connection is SSPI, and uses the local machine as default.
Provide 3 part aming in the SQL, as the connection defaults to master database.

All this , just to get linebreaks in the XML

## sample call
## .\exportXml "SELECT TOP 10 * FROM master.dbo.sysobjects" "C:\Dev\ps\" "sys.objects"

param($select,$path,$file,$instance)

if($select -eq $null -or $path -eq $null -or $file -eq $null)
{
"usage:
.\x `"SELECT FROM clause`" `"x:\path\`" `"filename`"";
exit;
}

## get the server name if instance not provided, defaults to local machine
if($instance -eq $null){ $srv = (gwmi Win32_ComputerSystem | Select-Object "Name").Name; }

## initialize variables and resources
$cn = new-object system.data.SqlClient.SqlConnection("Data Source=" + $instance + ";Integrated Security=SSPI;Initial Catalog=master");
$cmd = new-object system.Data.Sqlclient.SqlCommand($select, $cn);
$cmd.CommandType = [System.Data.CommandType]"Text"


$s = new-object System.Xml.XmlWriterSettings;
$s.Indent = $true;
$s.Encoding = [System.Text.Encoding]::Unicode;
$xw = [System.Xml.XmlWriter]::Create($path+$file+".xml",$s);


## cleanup resources
function cleanup()
{
if($cn -ne $null){$cn.Close();}
if($xw -ne $null){$xw.Close();}
}

## generic error trap
trap
{
$error[0];
cleanup;
exit;
}


## do the work
$xw.WriteStartElement($file);

$cn.Open()
$rdr = $cmd.ExecuteReader();
while($rdr.Read())
{
$xw.WriteStartElement("row");
$j = $rdr.FieldCount;
for($i=0;$i -lt $j;$i++)
{
$val = "";
if(!$rdr[$i].IsDBNull){ $val = $rdr[$i].ToString(); }
$xw.WriteElementString($rdr.GetName($i),$val);
#$rdr.GetName($i);$rdr[$i];
}
$xw.WriteEndElement();
}
$xw.WriteEndElement();

## create a file with the schema
$rdr.GetSchemaTable().WriteXml($path+$file+"schema.xml");


cleanup;


## To view the file and the schema file, uncomment
# "SCHEMA FILE: " + ($path+$file+"schema.xml"); Get-Content ($path+$file+"schema.xml");
# "DATA FILE: " + ($path+$file+"schema.xml"); Get-Content ($path+$file+".xml");


rockmoose
Go to Top of Page

jezemine
Master Smack Fu Yak Hacker

2886 Posts

Posted - 2007-04-09 : 23:14:37
cool. I've been meaning to start using monad but keep putting it off. old habits...

one other thing i thought of - if any column names have spaces or other non-xml friendly chars in them, this will break (because you can't have a space in an attribute element name).

I'm sure you don't have such poorly named columns though.


www.elsasoft.org
Go to Top of Page

rockmoose
SQL Natt Alfen

3279 Posts

Posted - 2007-04-10 : 04:56:12
The XmlWriter by default does some checking and throws an error in that case:

Exception calling "WriteElementString" with "2" argument(s): "Invalid name character in 'spa ce'. The ' ' character, hexadecimal value 0x20, cannot be included in a name."
At C:\dev\ps\exportXml.ps1:57 char:25
+ $xw.WriteElementString( <<<< $rdr.GetName($i),$val);


Yes, I like the powershell (former monad) since you can script with the .NET framework without having to compile stuff into tiny apps.

rockmoose
Go to Top of Page

jezemine
Master Smack Fu Yak Hacker

2886 Posts

Posted - 2007-04-10 : 09:25:24
exactly, that's what i meant by "break".


www.elsasoft.org
Go to Top of Page

rockmoose
SQL Natt Alfen

3279 Posts

Posted - 2007-04-10 : 13:22:28
quote:
Originally posted by jezemine

exactly, that's what i meant by "break".


www.elsasoft.org



LOL

Crash in a Null Reference, that's broken.
Displaying a friendly error message that can be easily fixed, that's not broken
Go to Top of Page
   

- Advertisement -