How to get started with creating a virtual filesystem?

I would like to create a new virtual volume with a virtual filesystem,
something that is visible as a drive, can be given a letter, and browsed in
"explorer" etc., but where all requests to access files are handled by me,
preferably delegated to a service in user space.

What kind of component do I actually need to write, at a minimum?

If I understand well, I would need to somehow create a fake volume and tell
Windows (I/O manager?) about it, register my own filesystem and make Windows
forward all requests to this fake volume to my filesystem instance, with
which I would interact via ioctl. Is that correct? Could it be simpler?

Is there a way to get there even faster with WDF? How about UMDF, does it
have support for this sort of functionality?

Don
Sun Mar 05 12:15:16 CST 2006

A filesystem requires the IFS kit. You should consider getting training
from OSR http://www.osr.com and expect to spend 1 to 2 calendar years (and 2
to 3 X that in man-years) to do the work.


--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply



"krzys" <krzys@discussions.microsoft.com> wrote in message
news:5D4C5E01-4316-42D3-976C-D8A8477E1D59@microsoft.com...
>I would like to create a new virtual volume with a virtual filesystem,
> something that is visible as a drive, can be given a letter, and browsed
> in
> "explorer" etc., but where all requests to access files are handled by me,
> preferably delegated to a service in user space.
>
> What kind of component do I actually need to write, at a minimum?
>
> If I understand well, I would need to somehow create a fake volume and
> tell
> Windows (I/O manager?) about it, register my own filesystem and make
> Windows
> forward all requests to this fake volume to my filesystem instance, with
> which I would interact via ioctl. Is that correct? Could it be simpler?
>
> Is there a way to get there even faster with WDF? How about UMDF, does it
> have support for this sort of functionality?
>
>

krzys
Sun Mar 05 12:37:26 CST 2006

Thank you for the response, but 2-4 man-years sounds absurd, I think that
perhaps I wasn't clear enough on what I want to do.

I don't want to implement an actual filesystem or disk driver, just forward
a bunch of requests such as create file or read file to my user-level
service. I don't want my fake filesystem to be used on "real" volumes or my
fake volume to be used for "real" filesystems. I want a fake, virtual drive
with a letter to pop up somewhere, that can accept open file and read/write
file requests and delegate them to user space without doing any processing.
The requests would actually be implemented in a way that has nothing to do
with physical devices, drives, buses, and I don't think I have any need to
mess with interrupts, cache, memory mapping and other low-level stuff. I
understand that all it takes should be to get a bunch of hook-ups and
wrappers in place.

I looked at the IFS kit, but I still don't see what exactly kind of entity
to write. A filesystem alone doesn't seem to be the right answer because, as
I said, I don't want it to be used with "real" volumes, only with my own
single "fake" volume.


"Don Burn" wrote:
> A filesystem requires the IFS kit. You should consider getting training
> from OSR http://www.osr.com and expect to spend 1 to 2 calendar years (and 2
> to 3 X that in man-years) to do the work.

Don
Sun Mar 05 14:00:35 CST 2006

It still is a filesystem, you are going to have to take the FAT sample rip
out most of the guts, and then do it with your service. This is not a small
project the smallest real filesystem I know of took almost 5 man years and
some have gone of 100 man years.

If you are not doing this for commercial work, the other possibility is to
make it a "nerwork filesystem" by using the Samba sources to start your
efforts. This does mean your requests go through the network, even if they
are to the same system, but you start with a solid piece of user space code
to handle things.


--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Remove StopSpam from the email to reply



"krzys" <krzys@discussions.microsoft.com> wrote in message
news:1BFBD192-BB14-4D38-926D-5B5254801C08@microsoft.com...
> Thank you for the response, but 2-4 man-years sounds absurd, I think that
> perhaps I wasn't clear enough on what I want to do.
>
> I don't want to implement an actual filesystem or disk driver, just
> forward
> a bunch of requests such as create file or read file to my user-level
> service. I don't want my fake filesystem to be used on "real" volumes or
> my
> fake volume to be used for "real" filesystems. I want a fake, virtual
> drive
> with a letter to pop up somewhere, that can accept open file and
> read/write
> file requests and delegate them to user space without doing any
> processing.
> The requests would actually be implemented in a way that has nothing to do
> with physical devices, drives, buses, and I don't think I have any need to
> mess with interrupts, cache, memory mapping and other low-level stuff. I
> understand that all it takes should be to get a bunch of hook-ups and
> wrappers in place.
>
> I looked at the IFS kit, but I still don't see what exactly kind of entity
> to write. A filesystem alone doesn't seem to be the right answer because,
> as
> I said, I don't want it to be used with "real" volumes, only with my own
> single "fake" volume.
>
>
> "Don Burn" wrote:
>> A filesystem requires the IFS kit. You should consider getting training
>> from OSR http://www.osr.com and expect to spend 1 to 2 calendar years
>> (and 2
>> to 3 X that in man-years) to do the work.
>

krzys
Mon Mar 06 12:42:29 CST 2006

Hmm, sounds discouraging... but thanks.

Assuming I'd give it a shot, along the lines of the FAT-ripping option...

How would I go about getting a fake volume that works only with my fake
filesystem, and vice versa, have my filesystem used only for the fake volume
I would create?

Would I need to create a fake disk device? Or could I just tell the I/O
manager to create a fake volume with a specific name and force it to use my
filesystem driver? How would I prevent my fake filesystem to be applied to
regular volumes?

There seems to be this split beteen filesystems and devices, but what if all
I want is a high-level entity that has folders and files, the semantics of
which may have nothing to do with blocks and sectors, and thus I actually
DON'T and CAN'T implement and expose anything that looks like a volume or a
storage device?

I'd like to somehow go around this rigid structure and only implement the
"file-level functionality" for my storage-like-looking entity, with the
minimum amount of placeholders and glue code to get things in place. Is this
at all possible?

In Linux, you can use filesystem interface for a variety of purposes. What
I'm trying to achieve on Windows is quite similar. I want files, but not data
blocks.

Krzys

"Don Burn" wrote:

> It still is a filesystem, you are going to have to take the FAT sample rip
> out most of the guts, and then do it with your service. This is not a small
> project the smallest real filesystem I know of took almost 5 man years and
> some have gone of 100 man years.
>
> If you are not doing this for commercial work, the other possibility is to
> make it a "nerwork filesystem" by using the Samba sources to start your
> efforts. This does mean your requests go through the network, even if they
> are to the same system, but you start with a solid piece of user space code
> to handle things.
>
>
> --
> Don Burn (MVP, Windows DDK)
> Windows 2k/XP/2k3 Filesystem and Driver Consulting
> Remove StopSpam from the email to reply
>
>
>
> "krzys" <krzys@discussions.microsoft.com> wrote in message
> news:1BFBD192-BB14-4D38-926D-5B5254801C08@microsoft.com...
> > Thank you for the response, but 2-4 man-years sounds absurd, I think that
> > perhaps I wasn't clear enough on what I want to do.
> >
> > I don't want to implement an actual filesystem or disk driver, just
> > forward
> > a bunch of requests such as create file or read file to my user-level
> > service. I don't want my fake filesystem to be used on "real" volumes or
> > my
> > fake volume to be used for "real" filesystems. I want a fake, virtual
> > drive
> > with a letter to pop up somewhere, that can accept open file and
> > read/write
> > file requests and delegate them to user space without doing any
> > processing.
> > The requests would actually be implemented in a way that has nothing to do
> > with physical devices, drives, buses, and I don't think I have any need to
> > mess with interrupts, cache, memory mapping and other low-level stuff. I
> > understand that all it takes should be to get a bunch of hook-ups and
> > wrappers in place.
> >
> > I looked at the IFS kit, but I still don't see what exactly kind of entity
> > to write. A filesystem alone doesn't seem to be the right answer because,
> > as
> > I said, I don't want it to be used with "real" volumes, only with my own
> > single "fake" volume.
> >
> >
> > "Don Burn" wrote:
> >> A filesystem requires the IFS kit. You should consider getting training
> >> from OSR http://www.osr.com and expect to spend 1 to 2 calendar years
> >> (and 2
> >> to 3 X that in man-years) to do the work.
> >
>
>
>

pavel_a
Mon Mar 06 14:50:27 CST 2006

"krzys" wrote:
> I want files, but not data blocks.

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/shellcc/platform/shell/programmersguide/shell_adv/namespaceextension/namespace.asp

Regards,
--PA

Sergei
Mon Mar 06 15:10:23 CST 2006

> There seems to be this split beteen filesystems and devices, but what if
all
> I want is a high-level entity that has folders and files, the semantics of
> which may have nothing to do with blocks and sectors, and thus I actually
> DON'T and CAN'T implement and expose anything that looks like a volume or
a
> storage device?
>
> I'd like to somehow go around this rigid structure and only implement the
> "file-level functionality" for my storage-like-looking entity, with the
> minimum amount of placeholders and glue code to get things in place. Is
this
> at all possible?

IMHO you don't need any volume/storage device. A file system driver can
function without that. AFAIK named pipes and network shares are implemented
that way. Although I've never tried to do that kind of thing myself, so I
don't know how hard it might be.
--
Sergei.

m
Mon Mar 06 17:27:11 CST 2006

What do you want to do with a filesystem, but without any data storage?
Dosen't that knid of defeat the purpose?

"krzys" <krzys@discussions.microsoft.com> wrote in message
news:D102F454-5813-4613-8881-6C46CD80DB10@microsoft.com...
> Hmm, sounds discouraging... but thanks.
>
> Assuming I'd give it a shot, along the lines of the FAT-ripping option...
>
> How would I go about getting a fake volume that works only with my fake
> filesystem, and vice versa, have my filesystem used only for the fake
> volume
> I would create?
>
> Would I need to create a fake disk device? Or could I just tell the I/O
> manager to create a fake volume with a specific name and force it to use
> my
> filesystem driver? How would I prevent my fake filesystem to be applied to
> regular volumes?
>
> There seems to be this split beteen filesystems and devices, but what if
> all
> I want is a high-level entity that has folders and files, the semantics of
> which may have nothing to do with blocks and sectors, and thus I actually
> DON'T and CAN'T implement and expose anything that looks like a volume or
> a
> storage device?
>
> I'd like to somehow go around this rigid structure and only implement the
> "file-level functionality" for my storage-like-looking entity, with the
> minimum amount of placeholders and glue code to get things in place. Is
> this
> at all possible?
>
> In Linux, you can use filesystem interface for a variety of purposes. What
> I'm trying to achieve on Windows is quite similar. I want files, but not
> data
> blocks.
>
> Krzys
>
> "Don Burn" wrote:
>
>> It still is a filesystem, you are going to have to take the FAT sample
>> rip
>> out most of the guts, and then do it with your service. This is not a
>> small
>> project the smallest real filesystem I know of took almost 5 man years
>> and
>> some have gone of 100 man years.
>>
>> If you are not doing this for commercial work, the other possibility is
>> to
>> make it a "nerwork filesystem" by using the Samba sources to start your
>> efforts. This does mean your requests go through the network, even if
>> they
>> are to the same system, but you start with a solid piece of user space
>> code
>> to handle things.
>>
>>
>> --
>> Don Burn (MVP, Windows DDK)
>> Windows 2k/XP/2k3 Filesystem and Driver Consulting
>> Remove StopSpam from the email to reply
>>
>>
>>
>> "krzys" <krzys@discussions.microsoft.com> wrote in message
>> news:1BFBD192-BB14-4D38-926D-5B5254801C08@microsoft.com...
>> > Thank you for the response, but 2-4 man-years sounds absurd, I think
>> > that
>> > perhaps I wasn't clear enough on what I want to do.
>> >
>> > I don't want to implement an actual filesystem or disk driver, just
>> > forward
>> > a bunch of requests such as create file or read file to my user-level
>> > service. I don't want my fake filesystem to be used on "real" volumes
>> > or
>> > my
>> > fake volume to be used for "real" filesystems. I want a fake, virtual
>> > drive
>> > with a letter to pop up somewhere, that can accept open file and
>> > read/write
>> > file requests and delegate them to user space without doing any
>> > processing.
>> > The requests would actually be implemented in a way that has nothing to
>> > do
>> > with physical devices, drives, buses, and I don't think I have any need
>> > to
>> > mess with interrupts, cache, memory mapping and other low-level stuff.
>> > I
>> > understand that all it takes should be to get a bunch of hook-ups and
>> > wrappers in place.
>> >
>> > I looked at the IFS kit, but I still don't see what exactly kind of
>> > entity
>> > to write. A filesystem alone doesn't seem to be the right answer
>> > because,
>> > as
>> > I said, I don't want it to be used with "real" volumes, only with my
>> > own
>> > single "fake" volume.
>> >
>> >
>> > "Don Burn" wrote:
>> >> A filesystem requires the IFS kit. You should consider getting
>> >> training
>> >> from OSR http://www.osr.com and expect to spend 1 to 2 calendar years
>> >> (and 2
>> >> to 3 X that in man-years) to do the work.
>> >
>>
>>
>>

Doron
Mon Mar 06 22:24:28 CST 2006

filesystems in *nix are the way many drivers expose data. that is not the
case in NT. you are trying to fit a square peg in a round hole. YOu will
have a much easier time if you just follow the std way of exposing data.

d

--
Please do not send e-mail directly to this alias. this alias is for
newsgroup purposes only.
This posting is provided "AS IS" with no warranties, and confers no rights.


"krzys" <krzys@discussions.microsoft.com> wrote in message
news:D102F454-5813-4613-8881-6C46CD80DB10@microsoft.com...
> Hmm, sounds discouraging... but thanks.
>
> Assuming I'd give it a shot, along the lines of the FAT-ripping option...
>
> How would I go about getting a fake volume that works only with my fake
> filesystem, and vice versa, have my filesystem used only for the fake
> volume
> I would create?
>
> Would I need to create a fake disk device? Or could I just tell the I/O
> manager to create a fake volume with a specific name and force it to use
> my
> filesystem driver? How would I prevent my fake filesystem to be applied to
> regular volumes?
>
> There seems to be this split beteen filesystems and devices, but what if
> all
> I want is a high-level entity that has folders and files, the semantics of
> which may have nothing to do with blocks and sectors, and thus I actually
> DON'T and CAN'T implement and expose anything that looks like a volume or
> a
> storage device?
>
> I'd like to somehow go around this rigid structure and only implement the
> "file-level functionality" for my storage-like-looking entity, with the
> minimum amount of placeholders and glue code to get things in place. Is
> this
> at all possible?
>
> In Linux, you can use filesystem interface for a variety of purposes. What
> I'm trying to achieve on Windows is quite similar. I want files, but not
> data
> blocks.
>
> Krzys
>
> "Don Burn" wrote:
>
>> It still is a filesystem, you are going to have to take the FAT sample
>> rip
>> out most of the guts, and then do it with your service. This is not a
>> small
>> project the smallest real filesystem I know of took almost 5 man years
>> and
>> some have gone of 100 man years.
>>
>> If you are not doing this for commercial work, the other possibility is
>> to
>> make it a "nerwork filesystem" by using the Samba sources to start your
>> efforts. This does mean your requests go through the network, even if
>> they
>> are to the same system, but you start with a solid piece of user space
>> code
>> to handle things.
>>
>>
>> --
>> Don Burn (MVP, Windows DDK)
>> Windows 2k/XP/2k3 Filesystem and Driver Consulting
>> Remove StopSpam from the email to reply
>>
>>
>>
>> "krzys" <krzys@discussions.microsoft.com> wrote in message
>> news:1BFBD192-BB14-4D38-926D-5B5254801C08@microsoft.com...
>> > Thank you for the response, but 2-4 man-years sounds absurd, I think
>> > that
>> > perhaps I wasn't clear enough on what I want to do.
>> >
>> > I don't want to implement an actual filesystem or disk driver, just
>> > forward
>> > a bunch of requests such as create file or read file to my user-level
>> > service. I don't want my fake filesystem to be used on "real" volumes
>> > or
>> > my
>> > fake volume to be used for "real" filesystems. I want a fake, virtual
>> > drive
>> > with a letter to pop up somewhere, that can accept open file and
>> > read/write
>> > file requests and delegate them to user space without doing any
>> > processing.
>> > The requests would actually be implemented in a way that has nothing to
>> > do
>> > with physical devices, drives, buses, and I don't think I have any need
>> > to
>> > mess with interrupts, cache, memory mapping and other low-level stuff.
>> > I
>> > understand that all it takes should be to get a bunch of hook-ups and
>> > wrappers in place.
>> >
>> > I looked at the IFS kit, but I still don't see what exactly kind of
>> > entity
>> > to write. A filesystem alone doesn't seem to be the right answer
>> > because,
>> > as
>> > I said, I don't want it to be used with "real" volumes, only with my
>> > own
>> > single "fake" volume.
>> >
>> >
>> > "Don Burn" wrote:
>> >> A filesystem requires the IFS kit. You should consider getting
>> >> training
>> >> from OSR http://www.osr.com and expect to spend 1 to 2 calendar years
>> >> (and 2
>> >> to 3 X that in man-years) to do the work.
>> >
>>
>>
>>

Maxim
Tue Mar 07 04:35:28 CST 2006

> filesystems in *nix are the way many drivers expose data. that is not the
> case in NT. you are trying to fit a square peg in a round hole. YOu will
> have a much easier time if you just follow the std way of exposing data.

Yes, for instance, WMI - effectively a Windows version of UNIX's /proc

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

pavel_a
Wed Mar 08 19:22:26 CST 2006

"Maxim S. Shatskih" wrote:
> > filesystems in *nix are the way many drivers expose data. that is not the
> > case in NT. you are trying to fit a square peg in a round hole. YOu will
> > have a much easier time if you just follow the std way of exposing data.
>
> Yes, for instance, WMI - effectively a Windows version of UNIX's /proc

With one little "but". Explorer is pretty, Wbemtest is ugly.
The best driver is no driver.

--PA

PGopalaKrishna
Wed Nov 15 04:01:02 CST 2006

"krzys" wrote:

> I would like to create a new virtual volume with a virtual filesystem,
> something that is visible as a drive, can be given a letter, and browsed in
> "explorer" etc., but where all requests to access files are handled by me,
> preferably delegated to a service in user space.
>
> What kind of component do I actually need to write, at a minimum?
>
> If I understand well, I would need to somehow create a fake volume and tell
> Windows (I/O manager?) about it, register my own filesystem and make Windows
> forward all requests to this fake volume to my filesystem instance, with
> which I would interact via ioctl. Is that correct? Could it be simpler?
>
> Is there a way to get there even faster with WDF? How about UMDF, does it
> have support for this sort of functionality?
>
>





Dear,

You can take a look at the "Ram Disk" sample from the WDK.

It is a virtual drive (in RAM). It does not have any physical device
attached to it. But it supports all the basic file funcitonality such as
directory/files structure, write, read etc..

However, any data stored in that drive would go away once we switch off the
power.

That is the nature of the Ram Drive. It seems you want to create some
similar drive for your purpose.

Take a look at the RamDisk sample and modify it to suit your needs.

All the best.

Yours,
P.Gopalakrishna.
http://www.geocities.com/krishnapg/

Dev
Wed Nov 15 15:38:02 CST 2006

Hi,

I installed the driver. I can see the driver is installed okay. How do I
used it? I do not find it anywhere?

Thanks.

"P.GopalaKrishna" wrote:

> "krzys" wrote:
>
> > I would like to create a new virtual volume with a virtual filesystem,
> > something that is visible as a drive, can be given a letter, and browsed in
> > "explorer" etc., but where all requests to access files are handled by me,
> > preferably delegated to a service in user space.
> >
> > What kind of component do I actually need to write, at a minimum?
> >
> > If I understand well, I would need to somehow create a fake volume and tell
> > Windows (I/O manager?) about it, register my own filesystem and make Windows
> > forward all requests to this fake volume to my filesystem instance, with
> > which I would interact via ioctl. Is that correct? Could it be simpler?
> >
> > Is there a way to get there even faster with WDF? How about UMDF, does it
> > have support for this sort of functionality?
> >
> >
>
>
>
>
>
> Dear,
>
> You can take a look at the "Ram Disk" sample from the WDK.
>
> It is a virtual drive (in RAM). It does not have any physical device
> attached to it. But it supports all the basic file funcitonality such as
> directory/files structure, write, read etc..
>
> However, any data stored in that drive would go away once we switch off the
> power.
>
> That is the nature of the Ram Drive. It seems you want to create some
> similar drive for your purpose.
>
> Take a look at the RamDisk sample and modify it to suit your needs.
>
> All the best.
>
> Yours,
> P.Gopalakrishna.
> http://www.geocities.com/krishnapg/

raj
Wed Jun 06 11:56:01 CDT 2007

It's not as bad, please look into nulmrx ddk sample. With a tdi client in
your driver you should be able to implement one.

"Pavel A." wrote:

> "Maxim S. Shatskih" wrote:
> > > filesystems in *nix are the way many drivers expose data. that is not the
> > > case in NT. you are trying to fit a square peg in a round hole. YOu will
> > > have a much easier time if you just follow the std way of exposing data.
> >
> > Yes, for instance, WMI - effectively a Windows version of UNIX's /proc
>
> With one little "but". Explorer is pretty, Wbemtest is ugly.
> The best driver is no driver.
>
> --PA
>

Don
Wed Jun 06 12:31:25 CDT 2007

NO IT IS WORSE, SORRY THE JOY'S OF FILESYSTEM WORK ARE ENOUGH TO ENSURE YOU
DON"T WANT TO DO ONE UNLESS YOU REALLY HAVE TO. Look at driver namespaces
or some other approach.

--
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply


"raj" <raj@discussions.microsoft.com> wrote in message
news:CF6CE6E0-27E8-475A-97C4-C6B94E573EF5@microsoft.com...
> It's not as bad, please look into nulmrx ddk sample. With a tdi client in
> your driver you should be able to implement one.
>
> "Pavel A." wrote:
>
>> "Maxim S. Shatskih" wrote:
>> > > filesystems in *nix are the way many drivers expose data. that is
>> > > not the
>> > > case in NT. you are trying to fit a square peg in a round hole. YOu
>> > > will
>> > > have a much easier time if you just follow the std way of exposing
>> > > data.
>> >
>> > Yes, for instance, WMI - effectively a Windows version of UNIX's /proc
>>
>> With one little "but". Explorer is pretty, Wbemtest is ugly.
>> The best driver is no driver.
>>
>> --PA
>>