General Network Error OpenXML

Hi

I am using OpenXML to import data from text file. While it is working

fine with small files...I am getting General Network error while

parsing file of 4.5MB. Could anyone suggest the possible cause for

this. What is interesting is that , I am able to load the same file

using another SQL server. Could someone tell me the possible reason for

this behaviour....

Thanks

AnandThis error typically denotes a problem at the network layer. Can you check the NT event logs of the server for any network card failures? What about the SQL errorlog? Does it have any suspicious messages?sql

Fuzzy lookup match issue

Hello,

I have a peculiar problem in my project. My project design is like this

The number in (...) are count of records.

File feed (1000)

|

|

Fuzzy Lookup

against Table2

|

|

Split Fz Lookup results

(_Similarity >= 0.60 && _Confidence >= 0.85)

| |

| |

| Write matches to Table1 (250)

|

Fuzzy Group

Remaining rows (750)

|

|

Split Fz Group results

| |

| |

Write Canonicals Write Dupes

to Table2 to Table1

(300) (450)

This is basically a customer de-dupification project.

The Table2 has the canonicals and Table1 has the dupes (of the canonicals).

I already have some data in these tables and the new data is matched against the existing data

in these tables and classified as new customers and duplicate customers.

In the above process one could notice that the rows identified as dupes of already exsting canonicals

by the Fuzzy Lookup task are written into the dupes table (Table1) and will not be processed further down

the line in the project.

But in my case I see that those matches identified by Fuzzy lookup are further being included in the

Fuzzy Grouping also.

When I run this in debug mode in BIDS, it shows the correct numbers as I have depicted in the

illustration above. But, after execution, when I query the tables it shows that all 1000 rows

went through Fuzzy Grouping.

Any thoughts?

Btw, is there anyway to upload attachments to the postings here?

I also tried introducing a Derived Column between the 'Split Fz Lookup Results' and 'Write matches to Table1' to write some string into one of the table columns. It did not.

Fuzzy Group Updates?

Hi there,

Quick Background: I have an SSIS package that reads data from a flat file then runs it through a Fuzzy Grouping component. The result of this Fuzzy Group is put into a SQL server 2005 table.

Question: Over time, the flat file will be adding new records (some that should be added to existing groups) and so I'll need to update my Fuzzy Group table to include these new records. Is there anyway to simply add these new records to the existing Fuzzy Group without changing all of the _key_out values? If I completely regenerate the Fuzzy Group table that will potentially give me different _key_out values correct?

Does this make sense?

Any help would be greatly appreciated!

>>" If I completely regenerate the Fuzzy Group table that will potentially give me different _key_out values correct?"

Correct.

wenyang

|||

Thanks for the reply!

Anyway to preserve the _key_out while still adding records to the groups? Sounds like a complete rebuild of the Fuzzy group is out of the question. Anyway to do this incrementally?

|||

Hi,

Yes, each time you run Fuzzy Grouping with a different set of input rows (or with a different threshold), it is possible that different groupings will result.

If you have run FG once and would like to keep the existing groups, one alternative would be to use Fuzzy Lookup for the incremental input rows. You would basically perform a fuzzy lookup against the output of FG and return the _key_out of the best matching row. You have thus effectively found a group for the new input row. If no match is found above the FL match threshold, then just assign a new unique _key_out to the input row to create a new group.

A slight problem with this approach is that over time all the incremental rows may not be grouped as well as they could be, as the clustering algorithm that Fuzzy Grouping uses to globally pick groupings is not being employed. At that point you may want to just rerun FG and switch to the new groupings.

We are considering adding a feature in the next version that will allow you keep all the old groupings intact.

Let us know if you have any more questions.

Regards,

-Kris

|||Thanks Kris, that will probably suffice for now. Yeah put that in the next version, the FG component is great but it doesn't have much use after the initial run because of this limitation.

Fulltext Thesaurus

My understanding of the fulltext search using the thesaurus function is that
it will not work until the TSxxx.XML file is populated (in my case the
tsENU.xml file.)
Is this true?
We rae migrating from SQL 2000 and I could have sworn this was working there
.
Maybe I am mistaken.
I do see where the MS Office products use the MSTH3AM.LEX & MSTH3BR.LEX
dictionary files to support the thesaurus lookup there. Can these files
somehow be migrated over to SQL Server 2005 for use there or somehow be read
and converted to an XML thesaurus?
Thomas MannHere's a thought......I believe MS Word was installed on the machine that
had SQL 2000 on it.
I know that when I set up a new desktop, spellcheck & thesaurus are not
available to other applications (i.e. Outlook Express) until after MS Word i
s
installed.
Could this be the reason the Thesaurus funtion worked on the previous
machine and not the machine running SQL 2005?
--
Thomas Mann|||I have opened a trouble ticket on this issue and will continue to post
updates as this progresses (so you don't have to.)
What I have found thus far is that the standard advice of removing the
commented xml code in the tsENU.xml file does not cut it. Once you do this,
the xml tries to reference the schema (tsSchema.xml) which is missing from
the SQL Server 2005 install. I located a copy of the file from an earlier
version (SQL 2000 SP3) and that solved my missing schema problem but still
the Thesaurus function would not work (even after adding the substituion
params in tsENU.xml.)
I burned one of my support incidents from my MSDN subscription and contacted
the support center. They checked several of the machines there and could not
find the missing schema file amoungst the installed files either.
Bottom line, it sounds like this might be an install problem (or an
undocumented feature.)
I will post the fix here when I find out more.
--
Thomas Mann

Full-Text Search

I am trying to create a full text index on a column in a table and the data type is Image, but the TYPE COLUMN FILE EXTENSION is in another table. there is no way of creating the type in the same table. the wizard will not allow me to pick the data type field from another table. is there a work around this issue?

Emadkb

Adding a column to the table is not an option?|||It is a fram work table and making any changes now would be impossible. Do you see any other solutions to this issue. I really appreciate your quick response. Emadkb|||If you are using SQL 2005, you can make an indexed view and build fulltext index on the view. However that do means you dup the storage for the image columns.|||Thank you so much....I was wishing there is another solution. emadkb|||

hey Feng Tian
I hope you could help me in FullText Search. No one else seems to be knowing about Fulltext Search.
I'm able to create Full text indexing catalogs, populate them etc.. when i login directly from the server using Sql Server Enterprise manager but when I try to use the same login (db owner) remotely from some other machine using SQL Server Enterprise Manager, I see disabled(stopped) Full-text Service and all full-text features disabled. So does it mean i cannot populate or create catalogs remotely using sql server enterprise manager.
I have tried this on two different servers and then accessing from two different remote/client machines. But the problem is there any way.

Server: Mssql 2000 server
Remote/Client system: Win XP service Pack2
Access Software: Microsoft Sql Server 2000 Enterprise Manager.

Thanks
Sattva

|||

Sattva,
The behievor you are experiencing with your SQL Server 2000 Enterprise Manager (EM) is typical and expected. The EM cannnot "see remote" services such as the "Microsoft Search" (MSSearch) service on remote servers when accessed across the internet or the server is registered as an IP address. However, SQL FTS is still functional and you can use the system stored procedures to remotely manage and populate the FT Catalogs, for example:

To Create/Remove the Existing Full-Text Table Index, Catalog
-- If Full-Text Index exists, DROP that Index,
-- If Full-Text Index does not exist, CREATE that Index.
--
use pubs
go
IF OBJECTPROPERTY ( object_id('pub_info'),'TableHasActiveFulltextIndex') = 1
BEGIN
print 'Table pub_info is Full-Text Enabled, dropping Full-Text Index & Catalog...'
EXEC sp_fulltext_table 'pub_info', 'drop'
EXEC sp_fulltext_catalog 'PubInfo', 'drop'
END
ELSE IF OBJECTPROPERTY ( object_id('pub_info'),'TableHasActiveFulltextIndex') = 0
BEGIN
print 'Table pub_info is NOT Full-Text Enabled, creating FT Catalog, Index & Activating...'
EXEC sp_fulltext_catalog 'PubInfo', 'create'
EXEC sp_fulltext_table 'pub_info', 'create', 'PubInfo', 'UPKCL_pubinfo'
EXEC sp_fulltext_column 'pub_info', 'pub_id', 'add'
EXEC sp_fulltext_column 'pub_info', 'pr_info', 'add'
EXEC sp_fulltext_table 'pub_info', 'activate'
END

EXEC sp_fulltext_catalog 'PubInfo', 'start_full' -- "Full Crawl"
Wait for crawl to complete
--
Confirm above results with:
--
SELECT pub_id, pr_info
FROM pub_info WHERE CONTAINS(pr_info, '"book*"')

See my blog entry "SQL Server 2000 Full-Text Search Resources and Links" for more SQL 2000 FTS information.

Hope that helps,
John

SQL Full Text Search Blog
http://jtkane.spaces.live.com/

|||

Thanks John. It was a great help. Now I hav it all working.

Cheers mate.